Weight compression method and device and data processing method and device
By quantizing the initial weight of the neural network model and compressing the number of bits, the target weight is obtained, and the data to be processed is shifted using the target weight, the problem of complexity of weight multiplication and addition operations in the neural network model is solved, and the effect of improving operation efficiency is achieved.
Patent Information
- Application Number
- CN202311362394.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-19
- Publication Date
- 2025-04-22
AI Technical Summary
The multiplication and addition of weights in neural network models are too complex, resulting in a degradation of system performance and it is difficult to effectively reduce computing resources and energy consumption.
By obtaining the original code corresponding to the initial weight of the neural network model, quantizing it into an encoding containing the target value, compressing the number of bits, obtaining the target weight, and shifting the data to be processed based on the target weight, simplifying the multiplication and addition operation.
It reduces the storage and transmission bandwidth of the neural network model, reduces the operation complexity, and improves the operation efficiency of the neural network model.
Smart Images
Figure CN119861898A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a weight compression method and device, and a data processing method and device. Background Art
[0002] During the application of neural network models, in order to reduce the chip area of the neural network model, the neural network model is usually quantized and the parameters in the neural network are converted from floating-point data to integer data.
[0003] In the prior art, after the feature data is sent to the neural network model, it needs to pass through each network layer of the neural network model, and after being processed by each network layer, it is output at the last layer. In the process of processing the feature data, each layer needs to quantize and perform multiplication and addition operations on the feature data and the weights of the neural network model. However, the multiplication and addition operations in the neural network model are too complicated, and a large amount of computing resources and energy are consumed during the operation process, which easily leads to a decrease in system performance. Summary of the invention
[0004] In view of this, the present application provides a weight compression method, comprising:
[0005] Obtaining a first code corresponding to an initial weight of a neural network model, where the first code is an original code corresponding to the initial weight;
[0006] quantizing the first code into a second code, wherein the second code includes at least one bit as a target value;
[0007] The second code is bit compressed to obtain a target weight, wherein the values of each group of valid position sequences in the target weight correspond to the positions of each target value in the second code.
[0008] In the above method, the first bit of each bit of the second code is a sign bit, and the remaining bits are value bits; the value bits have and include at least one target value, and the value of the second code corresponds to the value of the initial weight;
[0009] The step of compressing the second code by bits to obtain a target weight includes:
[0010] The target weight is formed based on the bit position of the target value in the sign bit and the value bit of the second code; wherein the first bit of the target weight is the sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and each group of valid position sequences is a binary number corresponding to the position number of the bit position of each target value in the second code.
[0011] A data processing method, comprising:
[0012] Obtaining a target weight of the neural network model, wherein the target weight is obtained by quantizing and compressing the original code corresponding to the initial weight in the neural network model;
[0013] Obtaining a third code corresponding to the data to be processed, wherein the third code is an original code corresponding to the data to be processed;
[0014] The third code is at least shifted based on the target weight to obtain a processing result corresponding to the data to be processed.
[0015] In the above method, the target weight includes a numerical sequence; wherein the first bit of the numerical sequence is a sign bit, which is used to represent the sign bit of the original code corresponding to the initial weight; the other bits are numerical bits, and the numerical bits include a plurality of groups of valid position sequences corresponding to the plurality of target values and arranged in order, and the numerical value of each group of valid position sequences represents the bit position of the corresponding target value in the first code.
[0016] In the above method, the step of performing at least a shift process on the third code based on the target weight to obtain a processing result corresponding to the data to be processed includes:
[0017] In a case where both the target weight and the third code are not zero, determining each shift number of the third code based on the target weight, wherein the shift number is a value of each group of valid position sequences in the target weight;
[0018] Shifting the numerical bits in the third code respectively based on the shift numbers to obtain shift results;
[0019] Calculating the value of the sign bit of the target weight and the value of the sign bit of the third code to obtain a sign value;
[0020] Based on the symbol value and the shift result, a processing result corresponding to the data to be processed is generated.
[0021] In the above method, the step of calculating the value of the sign bit of the target weight and the value of the sign bit of the third encoding to obtain the sign value includes:
[0022] An XOR process is performed on the value of the sign bit of the target weight and the value of the sign bit of the third code to obtain a sign value.
[0023] In the above method, the step of obtaining the third code corresponding to the data to be processed includes:
[0024] When the operation condition of the target network layer of the neural network model meets the first condition, obtaining a third code corresponding to the data to be processed;
[0025] The first condition indicates that the target network layer supports native code operation.
[0026] The above method, after obtaining the processing result corresponding to the data to be processed, further includes:
[0027] Result data corresponding to the processing result is obtained, where the result data is the complement of the processing result.
[0028] A weight compression device, comprising:
[0029] A first acquisition unit, used to acquire a first code corresponding to an initial weight of a neural network model, wherein the first code is an original code corresponding to the initial weight;
[0030] A quantization unit, configured to quantize the first code into a second code, wherein the second code includes at least one bit as a target value;
[0031] A compression unit is used to perform bit compression on the second code to obtain a target weight, wherein the values of each group of valid position sequences in the target weight correspond to the positions of each target value in the second code.
[0032] A data processing device, comprising:
[0033] A second acquisition unit is used to obtain a target weight of the neural network model, wherein the target weight is obtained by quantizing and compressing the original code corresponding to the initial weight in the neural network model;
[0034] A third acquisition unit, used to obtain a third code corresponding to the data to be processed, wherein the third code is an original code corresponding to the data to be processed;
[0035] A processing unit is used to perform at least a shift process on the third code based on the target weight to obtain a processing result corresponding to the data to be processed. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0037] Figure 1 A method flow chart of a weight compression method provided in an embodiment of the present application;
[0038] Figure 2 A device structure diagram of a weight compression device provided in an embodiment of the present application;
[0039] Figure 3 A method flow chart of a data processing method provided in an embodiment of the present application;
[0040] Figure 4 A flowchart of another method of a data processing method provided in an embodiment of the present application;
[0041] Figure 5 A device structure diagram of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0043] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0044] The present application can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, etc.
[0045] The embodiment of the present application provides a weight compression method, which can be applied to a variety of system platforms, and its execution subject can be a processor of a computer terminal or various mobile devices. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:
[0046] S101: Obtain a first code corresponding to an initial weight of a neural network model.
[0047] The initial weight is a binary complement code, and the first code is the original code corresponding to the initial weight.
[0048] In the present application, the neural network model is a trained neural network model, and the initial weights in the neural network model are in the form of complement codes. Before compressing the weights, the complement codes are converted into original codes.
[0049] It should be noted that if the value corresponding to the initial weight is a positive number, the original code and the complement code of the initial weight are consistent; if the value corresponding to the initial weight is a negative number, the initial weight needs to be inverted and then added to 1.
[0050] It should also be noted that the initial weights in the neural network model can be int8, int16 or int32 or other int type data. Therefore, the value range of the initial weights in the neural network model is -128 to 127, and the original code corresponding to this value range is: 11111111 to 01111111. Among them, the first bit is the sign bit, and the sign bit value "1" represents a negative number, and "0" represents a positive number.
[0051] Furthermore, the binary original code of the value -128 is "10000000". Since the weights in the neural network model are quantized into binary numbers in the form of int8, and the first digit in int8 needs to be used as the sign bit of the value, the binary original code of -128 in the form of int8 is not "110000000", but "10000000", where the first digit "1" in "10000000" indicates that the sign of the value corresponding to the binary original code is a negative number.
[0052] S102: quantize the first code into a second code.
[0053] The second code includes at least one target value.
[0054] It should be noted that the target value is "1". The second code can be a one-hot code (one-hot) containing only one target value "1", or a binary code (n-hot) containing multiple target values "1". Each bit of the second code includes a sign bit and a value bit, wherein the first bit of each bit of the second code is a sign bit, and the remaining bits are value bits. The sign bit can use "+" or "-" to represent the sign of the second code.
[0055] When the second code includes at least one target value, the value of the second code is a power of 2 or a sum of multiple powers of 2. For example, the value of the second code 01000000 is 64=2 6 , the value of the second code 00110000 is 48=2 4 +2 5 , the value of the second code 01001010 is 74=2 6 +2 3 +21 The number of target values of the second code is less than a preset value, and the preset value N≥1. When the first code is quantized into the second code, a binary value close to the value of the first code is selected as the second code, or only the target value "1" of the highest bit position in the initial weight is retained, and the values of the remaining value bits are all "0".
[0056] For example: the first code is 87 = (01010111) 补 , the first code needs to be converted into a second code containing two target values, and the sum of the two powers of 2 closest to the value of the first code is 80=2 6 +2 4 , so the second code obtained after quantizing the first code is +1010000. Since 87 is a positive number, the first bit of the first code is the sign bit, and the value "0" is used to represent the sign "+".
[0057] For another example, the first code is 48=(00110000) 补 , it is necessary to convert the first code into a second code containing a target value, retaining the target value "1" at the fifth bit position, so the second code obtained after quantizing the first code is +01000000.
[0058] S103: Compress the second code by bits to obtain a target weight.
[0059] The values of each group of valid position sequences in the target weight correspond to the positions of the target values in the second code. The number of bits in the target weight is less than the number of bits in the second code. For example, int8 data is compressed into 4-bit data int4.
[0060] Specifically, the specific implementation method for compressing the number of bits of the second code is: based on the sign bit of the second code and the bit position of the target value "1" in the numerical bit, a target weight is formed; wherein the first bit of the target weight is the sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and each group of valid position sequences is a binary number corresponding to the position number of the bit position of each target value "1" in the second code.
[0061] For example, if the first code is 80=01010000, the rightmost bit is the 0th bit, and the 4th bit position and the 6th bit position are in the bit positions from right to left, so the second code is 01101000. The first bit from left to right is the sign bit, and the valid position sequences corresponding to the other numerical bits are 110 and 100 respectively.
[0062] In the method provided in the embodiment of the present application, after the training of the neural network model is completed, the first code corresponding to the initial weight of the neural network model is obtained, and then the first code is quantized into a second code including at least one bit as the target value. Based on the bit position of the target value "1" in the sign bit and the value bit of the second code, the sign in the sign bit of the second code is converted into a value consistent with the sign bit in the initial weight, and the binary number corresponding to the position number of the position where the target value is located in the value bit of the second code is used as the value bit of the target weight.
[0063] In the present application, in the int8 data, if the first code is quantized into a target value with only one bit, and the second code is of int8 type, the corresponding binary numbers are -0000000, -0000001, -0000010, -0000100, -0001000, -0010000, -0100000, -1000000, +0000000, +0000001, +0000010, +0000100, +0001000, +0010000, +1000000, where the bits where the sign "+" and the sign "-" are located are sign bits, and the remaining bits are value bits. In the process of compressing the above-mentioned second code, the sign of the sign bit in the second code is converted into the corresponding numerical value (for example: the sign "+" is converted into "0", and the sign "-" is converted into "1"), and then the bit position of each numerical bit is compressed according to the bit position of the target numerical value "1" in each numerical bit, and the corresponding binary numbers in the obtained target weights are 1111, 1000, 1001, 1010, 1011, 1100, 1101, 1110, 0111, 0000, 0001, 0010, 0011, 0100, 0101, 0110, among which, since the target weight obtained after compression of -0000001 is 1000, 1111 is expressed as -0000000, and since the target weight obtained after compression of +0000001 is 0000, 0111 is expressed as +00000000.
[0064] In the present application, taking the value -32 as an example, its corresponding binary original code is 00100000. In the int8 form, the original code of -32 is 10100000, where the first bit "1" represents the symbol "-". After obtaining the original code of -32 in the int8 form, it is converted into a one-hot encoding, that is, converted into a numerical value in which at most one bit is "1". The one-hot encoding corresponding to -32 is -0100000. Since the target value "1" is located at the 5th bit position of each numerical bit (counting from right to left, the first bit is the 0th bit), after compressing -0100000, the target weight obtained is 1101, where the first bit 1 represents a negative number, and the binary number 101 of the subsequent three numerical bits corresponds to the decimal value 5, which corresponds to the 5th bit of the target value "1" in the second encoding (one-hot encoding).
[0065] Optionally, the second code may be compressed according to the power value of the power of 2 corresponding to the second code. Since there is at most one bit of the target value "1" in the second code, the value of each numerical bit of the second code is a power of 2 (for example, the power of 2 corresponding to 0000001 is 20, and the power of 2 corresponding to 0000010 is 2 1 , the power of 2 corresponding to 0000100 is 2 2 ), the bit position of the target value in the corresponding second code is the power value of 2. Taking the second code with only one target value as an example, the second code in the form of int8 is compressed into a 4-bit representation of the target weight: the highest bit is the sign bit, 1 represents a negative number, and 0 represents a positive number; the lower three bits are the value bits, and the value of the value bit represents the power value of 2 corresponding to the value of the second code. Among them, because the value range of int8 (the first bit is the sign bit, and the last 7 bits are the value bits) is -128 to 127, there is no 2 7 , so 2 7 =0111 represents 0. For example, if the second code is -32=10100000, the corresponding power of 2 is -2 5 , then the target weight obtained after compressing the second code -32 is -5 = 1101. Taking the second code containing multiple target values as an example, the second code in the form of int8 can be compressed into a 7-bit representation of the target weight as follows: the highest bit is the sign bit, 1 represents a negative number, and 0 represents a positive number; the lower three bits are the value bits, and the value of the value bit represents the power value of the power of 2 corresponding to the value of the second code.
[0066] In the method provided in the embodiment of the present application, after the training of the neural network model is completed, the weights in the neural network model are compressed to simplify the process of calculating the feature data by applying the compressed weights. In the neural network model, quantizing and compressing the weights after converting them into original codes can not only reduce the storage and transmission bandwidth of the neural network model, but also reduce the complexity of the operation when performing multiplication and addition operations on the compressed weights, thereby speeding up the operation efficiency of the neural network model.
[0067] and Figure 1 Corresponding to the method described above, the embodiment of the present application also provides a weight compression device for Figure 1 The specific implementation of the method in the embodiment of the present application, the weight compression device provided in the embodiment of the present application can be applied to a computer terminal or various mobile devices, and its structural diagram is as follows Figure 2 As shown, specifically including:
[0068] A first acquisition unit 201 is used to acquire a first code corresponding to an initial weight of a neural network model, where the first code is an original code corresponding to the initial weight;
[0069] A quantization unit 202, configured to quantize the first code into a second code, wherein the second code includes at least one bit as a target value;
[0070] The compression unit 203 is used to perform bit compression on the second code to obtain a target weight, wherein each group of valid position sequences in the target weight corresponds to the position of each target value in the second code.
[0071] In the device provided in the embodiment of the present application, the first bit of each bit of the second code is a sign bit, and the remaining bits are value bits; the value bits include at least one target value, and the value of the second code corresponds to the value of the initial weight;
[0072] The compression unit 203 performs bit compression on the second code to obtain a target weight, and is specifically used to: form the target weight based on the sign bit of the second code and the bit position of the target weight in the numerical bit; wherein the first bit of the target weight is the sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and the numerical bit is a binary number corresponding to the position number of the bit position of the target weight in the second code.
[0073] In the device provided by the present application, after completing the training of the neural network model, the first code corresponding to the initial weight of the neural network model is obtained, and then the first code is quantized into a second code containing at least one bit as the target value. Based on the bit position of the target value "1" in the sign bit and the value bit of the second code, the sign in the sign bit of the second code is converted into a value consistent with the sign bit in the initial weight, and the binary number corresponding to the position number of the position where the target value is located in the value bit of the second code is used as the value bit of the target weight. The weights in the neural network model are compressed to simplify the process of calculating the feature data by applying the compressed weights.
[0074] Based on the weight compression method and device provided in the above embodiment, the present application also provides a data processing method, the method flow chart of which is as follows: Figure 3 As shown, specifically including:
[0075] S301: Obtain target weights of a neural network model.
[0076] The target weight is obtained by quantizing and compressing the original code corresponding to the initial weight in the neural network model.
[0077] Specifically, after the neural network model is trained, the initial weights in the neural network model are converted into a first code, which is the original code of the initial weights. The first code is then quantized and compressed to obtain the target weights.
[0078] In the present application, the target weight includes a numerical sequence; wherein the first bit of the numerical sequence is a sign bit, which is used to represent the sign bit of the original code corresponding to the initial weight; the other bits are numerical bits, and the numerical bits include multiple groups of valid position sequences corresponding to the multiple target values and arranged in order, and the numerical value of each group of valid position sequences represents the bit position of the corresponding target value in the first code.
[0079] It should be noted that the numerical sequence of the target weight is a binary number, in which the first digit with the numerical value "0" or "1" represents the sign of the target weight, "0" represents the sign "+", and "1" represents the sign "-". The sign bit of the target weight is consistent with the initial weight and the sign bit of the original code corresponding to the initial weight. The other bits of the target weight except the first bit are numerical bits, and the number of bits of the numerical bits of the target weight is less than the number of bits of the initial weight.
[0080] It should also be noted that the initial weight is int8 data, and the original code corresponding to the initial weight is compressed to a 4-bit target weight after quantization. The process of compressing the weight of the neural network model refers to the weight compression method mentioned above, and will not be repeated here.
[0081] The target weights can be obtained by processing the initial weights provided by the model trainer and configuring them in the model when the network is deployed; or by converting the initial weights each time the model is used to continue data processing. If the target weights are configured in the model after the initial weights are converted into the original code and then quantized and compressed when the network is deployed, then when the neural network model is used for data processing, the target weights are directly applied to implement the data processing process of the neural network model. If the target weights are obtained after the initial weights are converted into the original code and then quantized and compressed each time the neural network model is used, then before the neural network model is used for data processing, the initial weights in the neural network model are first quantized and compressed, and after obtaining the target weights, the target weights are applied to implement the data processing process of the neural network model.
[0082] S302: Obtain a third code corresponding to the data to be processed.
[0083] The third code is the original code corresponding to the data to be processed.
[0084] It should be noted that the data to be processed is the feature data input into the neural network model, or the feature data preprocessed after input into the neural network model. Before applying the weights in the neural network model to perform multiplication and addition operations on the feature data, the feature data or the preprocessed feature data is converted into original code.
[0085] The data to be processed may specifically be text, image, video, or audio data.
[0086] S303: Perform at least a shift process on the third code based on the target weight to obtain a processing result corresponding to the data to be processed.
[0087] It should be noted that the shift number for shifting the third code is determined according to the value of the target weight, and the third code is shifted according to the shift number.
[0088] Specifically, the shift processing of the third code is to shift the numerical bits in the third code, and the shifting method is to shift the numerical bits to the left according to the shift number. If the target weight has only one set of valid position sequences, the numerical value of the target weight is used as the shift number to shift the third code once, and the result after the shift is the processing result corresponding to the data to be processed. If the target weight contains multiple sets of valid position sequences, the numerical value of each set of valid position sequences is used as the shift number to shift the third code multiple times and then add them to obtain the processing result corresponding to the data to be processed.
[0089] The processing result obtained by the present application is similar to the result of the multiplication and addition operation. Among them, the present application only needs to shift the data to be processed to obtain the final result, while the prior art requires multiple multiplication and addition operations to obtain the final result. Taking the initial weight as 16=00010000 and the data to be processed as 20=00010100 as an example, when the initial weight and the data to be processed are multiplied (00010000×00010100), the calculation process of the multiplication and addition algorithm of the prior art is as follows:
[0090] 00000000+00000000+00000000+00000000+101000000
[0091] =000101000000
[0092] It can be seen that the calculation process of the multiplication using the prior art requires multiple addition operations. The larger the initial weight, the longer the addition process. If the target weight 4=0100 obtained after quantizing and compressing the initial weight 16=00010000 is used to perform shift processing on the processed data 20=00010100 based on the method provided by S301 to S303 of this embodiment, the specific calculation process is as follows:
[0093] 00010100<<4=000101000000
[0094] It can be seen from the above example that in the process of applying the existing technology to perform multiplication calculations on the initial weights and the data to be processed, a large number of additions are required to obtain the final result. After the initial weights are quantized and compressed, only the data to be processed needs to be shifted to obtain the final result. The data processing method of the present application can greatly reduce the demand for adders in the neural network model during the calculation process, simplify the complex multiplication operations into shift operations, and the hardware implementation is simple. While saving chip area and power consumption, it also improves the efficiency of data processing by the neural network model.
[0095] It should also be noted that, in addition to shifting the numerical bit of the third code, the numerical value of the sign bit in the third code needs to be processed. The processing of the numerical value of the sign bit in the third code is to calculate the numerical value of the sign bit of the target weight and the numerical value of the sign bit of the third code to obtain a sign numerical value, which is used as the numerical value of the sign bit of the processing result.
[0096] The calculation of the value of the sign bit of the target weight and the value of the sign bit of the third code is specifically to perform an XOR process on the value of the sign bit of the target weight and the value of the sign bit of the third code.
[0097] In the method provided in the embodiment of the present application, the target weight of the neural network model and the third code corresponding to the data to be processed are obtained, the shift number of the three codes is determined according to the target weight, and the numerical bit in the third code is shifted according to the shift number to obtain the shift result; at the same time, the numerical value of the sign bit of the target weight and the data of the sign bit of the third code are XORed to obtain the symbol value. According to the symbol value and the shift result, the processing result corresponding to the data to be processed is generated, the first bit in the processing result is the sign bit, the value of the sign bit is the symbol data, and the other bits are the numerical bits, and the numerical bits are the shift result. Based on the method provided in the embodiment of the present application, the multiplication and addition process in the neural network model is replaced by shift processing, so as to simplify the data processing process and improve the processing speed of the neural network model.
[0098] In the embodiment of the present application, in addition to the shift processing, the process of processing the third code based on the target weight also needs to process the value of the sign bit of the third code. Figure 4 The specific process of performing at least a shift process on the third code based on the target weight and obtaining a processing result corresponding to the data to be processed includes:
[0099] S401: When both the target weight and the third code are not zero, determine each shift number of the third code based on the target weight.
[0100] The number of shifts is the value of each valid position sequence in the target weight.
[0101] It is understandable that the target weight and the third encoding are not zero, which means that the value of the target weight value bit or the value of the third encoding value bit is not zero. If any one of the target weight and the data to be processed is zero, the product of the two is zero, and there is no need to continue calculating. If the target weight and the data to be processed are not zero, then the number of shifts is determined according to the target weight.
[0102] S402: Shift the numerical bits in the third code based on the shift amounts respectively and then add them to obtain a shift result.
[0103] It should be noted that each bit of the third code includes a sign bit and a value bit. Among them, the first bit in each bit is a sign bit, and the value "0" in the sign bit represents the symbol "+", that is, it means that the value of the third code is a positive number, and the value "1" represents the symbol "-", that is, it means that the value of the third code is a negative number. The other bits in the third code except the first bit are value bits, and the value bits are the original code corresponding to the data to be processed.
[0104] Optionally, before the numerical bits in the third code are shifted, each numerical bit may be padded, that is, zeros are padded on the left side of each numerical bit so that the number of bits of each numerical bit after zero padding reaches the target number.
[0105] In this application, if there is only one shift number, the calculation method for the shift processing is as follows:
[0106] c={8'b0,a}<<b
[0107] Wherein, c is the shift result, a represents each numerical bit in the third code, 8'b0 represents that 8 bits of zero are added to the left of each numerical bit, and b is the shift number.
[0108] If there are multiple shifting bits, the calculation method for shifting is as follows:
[0109] c={8'b0,a}<<b+{8'b0,a}<<d
[0110] Where b and d are the shift numbers.
[0111] S403: Calculate the value of the sign bit of the target weight and the value of the sign bit of the third code to obtain a sign value.
[0112] The method for calculating the value of the sign bit of the target weight and the value of the sign bit of the third code is to perform an XOR process on the value of the sign bit of the target weight and the value of the sign bit of the third code.
[0113] In this application, the calculation method for XOR processing is as follows:
[0114] sign_c=sign_aXORsign_b
[0115] Among them, sign_c is the sign value, sign_a is the value of the sign bit of the third encoding, and sign_b is the value of the sign bit of the target weight.
[0116] S404: Generate a processing result corresponding to the data to be processed based on the symbol value and the shift result.
[0117] It should be noted that the processing result of the data to be processed is obtained by combining the sign value and the shift result. In each bit of the processing result, the first bit is the sign bit, the value of the sign bit is the sign value obtained according to S403, and the other bits are value bits, and the value in each value bit is the shift result obtained according to S402.
[0118] In this application, the process of obtaining the processing result corresponding to the data to be processed is described according to the following embodiments:
[0119] (1) The third coding and target weights are both negative;
[0120] Take the data to be processed as -38 and the target weight as -2 as an example. The data to be processed is -38 = (11011010) 补码 The third code obtained after conversion into the original code is -38 = (10100110) 原码 , the initial weight is -2, and the corresponding second code is 10000010. The value "1" in the numerical bit is at the first bit position. Therefore, the target weight obtained after compression is represented as -1=1001. Because both data are not 0, shift and XOR operations are required.
[0121] The shift result of the numerical bit is: 000000000100110<<1=000000001001100=76;
[0122] The sign value of the sign bit is: 1XOR1=0;
[0123] The final processing result is: (0000000001001100) 原码 =76.
[0124] (2) The third coding and target weights are both positive;
[0125] Take the data to be processed 40 and the target weight 4 as an example. Since the original code and the complement code of a positive number are the same, the data to be processed 40 = (0101000) 补码 The third code obtained after conversion into the original code is 40 = (0101000) 原码 The initial weight is 4, corresponding to the second code 4=00000100. The value "1" in the numerical bit is at the second bit position. Therefore, the target weight obtained after compression is represented as 2=0010. Because both data are not 0, shift and XOR operations are required.
[0126] The result of shifting the numeric bit is: 000000000101000<<2=000000010100000=160
[0127] The sign value of the sign bit is: 0XOR0=0
[0128] The final processing result is: (000000010100000) 原码 =160.
[0129] (3) The third code is a negative number and the target weight is a positive number;
[0130] Take the data to be processed as -38 and the target weight as 2 as an example. The data to be processed is -38 = (11011010) 补码 The third code obtained after conversion into the original code is -38 = (10100110) 原码The initial weight is 2, and the corresponding second code is 2=00000010. The value "1" in the numerical position is in the first bit position. Therefore, the target weight obtained after compression is represented as 1=0001, and shift and XOR operations are required.
[0131] The shift result of the numerical bit is: 0000000000100110<<1=0000000001001100=76;
[0132] The sign value of the sign bit is: 1XOR0=1;
[0133] The final processing result is: (1000000001001100) 原码 =-76.
[0134] (4) The third code is a positive number and the target weight is a negative number;
[0135] Take the case where the data to be processed is 38 and the target weight is -2 as an example. The data to be processed is 38 = (10100110) 补码 The third code obtained after conversion into the original code is -38 = (10100110) 原码 The initial weight is 2, and the corresponding second code is 2=10000010. The value "1" in the numerical position is in the first bit position. Therefore, the target weight obtained after compression is represented as -1=1001, and shift and XOR operations are required.
[0136] The shift result of the numerical bit is: 0000000000100110<<1=0000000001001100=76;
[0137] The sign value of the sign bit is: 0XOR1=1;
[0138] The final processing result is: (1000000001001100) 原码 =-76.
[0139] (5) The third code is a positive number, and the target weight is a positive number;
[0140] Take the data to be processed 40 and the target weight 6 as an example. Since the original code and the complement code of a positive number are the same, the data to be processed 40 = (0101000) 补码 The third code obtained after conversion into the original code is 40 = (0101000) 原码, the initial weight is 6, corresponding to the second code 6=00000110, the value "1" in the numerical position is at the first bit position and the second bit position, so the target weight obtained after compression is represented as 0010001, and the valid position sequences of each group in the target weight are 010 and 001 respectively. The first value "0" in the target weight indicates that the target weight is a positive number. Because the target weight and the data to be processed are not 0, shift and XOR operations are required.
[0141] The result of shifting the numeric bits is:
[0142] 000000000101000<<2+000000000101000<<1=000000011110000=240
[0143] The sign value of the sign bit is: 0XOR0=0
[0144] The final processing result is: (000000011110000) 原码 =240.
[0145] In the method provided by the present application, after the neural network model completes training, the initial weight of the neural network model is converted into the original code to obtain the first code. The first code is quantized into the second code so that the second code contains at least one numerical bit with a target value of "1". The second code is compressed according to the bit position of the target value "1" in the second code to obtain the target weight. After the feature data is input into the neural network model, the weight needs to be applied to process the feature data before obtaining the data to be processed. The data to be processed is the feature data input into the neural network model or the data after the feature data is preprocessed. The data to be processed is converted into the original code to obtain the third code. When both the target weight and the third code are not zero, the shift number of the third code is determined based on the target weight, and the numerical bit of the third code is shifted according to the shift number to obtain the shift result; at the same time, the numerical value of the sign bit of the target weight and the numerical value of the sign bit of the third code are XORed to obtain the symbol value; the symbol value and the shift result are combined to obtain the processing result of the data to be processed.
[0146] It should be noted that the neural network model includes at least one network layer for processing feature data, and each network layer corresponds to a weight for processing feature data. After the neural network model is trained, the original code corresponding to the initial weights of all network layers is quantized and compressed to obtain the target weight corresponding to each network layer.
[0147] It should also be noted that in the neural network model, there may be some network layers that do not support original code operations. Therefore, after the training of the neural network model is completed, only the original codes corresponding to the initial weights corresponding to the network layers that support original code operations are quantized and compressed. For the initial weights that have not been converted into original codes, when the initial weights are applied to process the data to be processed, there is no need to convert the data to be processed into original codes. After the initial weights are quantized and compressed, the compressed weights are applied to the data to be processed for shift processing to obtain the processing results corresponding to the data to be processed.
[0148] It can be understood that the specific implementation method of obtaining the third encoding corresponding to the data to be processed is: when the operation conditions of the target network layer of the neural network model meet the first condition, the third encoding corresponding to the data to be processed is obtained; wherein the first condition represents that the target network layer supports original code operation.
[0149] Furthermore, after obtaining the processing result, the processing result is converted into a complement code to obtain result data corresponding to the processing result. If the target network layer is the last layer of the neural network model, the neural network model can directly output the result data; if the target network layer is not the last layer of the neural network model, and the operation condition of the next network layer meets the first condition, the result data is used as the data to be processed of the next network layer, or after preprocessing the result data, the preprocessing result is used as the data to be processed of the next network layer.
[0150] In the data processing method provided in the embodiment of the present application, the processing result of the data to be processed is obtained by shifting the data to be processed by the target weight, without the need to perform multiplication and addition operations on the data, thereby simplifying the data processing process and improving the processing speed of the neural network model.
[0151] The specific implementation processes and derivative methods of the above-mentioned embodiments are all within the protection scope of this application.
[0152] and Figure 1 Corresponding to the method described above, the embodiment of the present application further provides a data processing device for Figure 1 In order to implement the method in the embodiment of the present application, the data processing device provided in the embodiment of the present application can be applied to a computer terminal or various mobile devices, and its structural diagram is as follows: Figure 5 As shown, specifically including:
[0153] The second acquisition unit 501 is used to obtain the target weight of the neural network model, where the target weight is obtained by quantizing and compressing the original code corresponding to the initial weight in the neural network model;
[0154] A third acquisition unit 502 is used to obtain a third code corresponding to the data to be processed, where the third code is the original code corresponding to the data to be processed;
[0155] The processing unit 503 is used to perform at least a shift process on the third code based on the target weight to obtain a processing result corresponding to the data to be processed.
[0156] In the device provided in an embodiment of the present application, the target weight includes a numerical sequence; wherein the first bit of the numerical sequence is a sign bit, which is used to represent the sign bit of the original code corresponding to the initial weight; the other bits are numerical bits, and the numerical bits include multiple groups of valid position sequences corresponding to the multiple target values and arranged in order, and the numerical value of each group of valid position sequences represents the bit position of the corresponding target value in the first code.
[0157] In the device provided in the embodiment of the present application, the processing unit 503 performs at least a shift process on the third code based on the target weight to obtain a processing result corresponding to the data to be processed, which is specifically used for:
[0158] In a case where both the target weight and the third code are not zero, determining each shift number of the third code based on the target weight, wherein the shift number is a value of each group of valid position sequences in the target weight;
[0159] Based on each of the shift numbers, the numerical bits in the third code are shifted and then added to obtain a shift result;
[0160] Calculating the value of the sign bit of the target weight and the value of the sign bit of the third code to obtain a sign value;
[0161] Based on the symbol value and the shift result, a processing result corresponding to the data to be processed is generated.
[0162] In the device provided in the embodiment of the present application, the processing unit 503 calculates the value of the sign bit of the target weight and the value of the sign bit of the third encoding to obtain a sign value, which is specifically used for:
[0163] An XOR process is performed on the value of the sign bit of the target weight and the value of the sign bit of the third code to obtain a sign value.
[0164] In the device provided in the embodiment of the present application, the third acquisition unit 502 obtains the third code corresponding to the data to be processed, which is specifically used for:
[0165] When the operation condition of the target network layer of the neural network model meets the first condition, obtaining a third code corresponding to the data to be processed;
[0166] The first condition indicates that the target network layer supports native code operation.
[0167] The device provided in the embodiment of the present application also includes:
[0168] The fourth acquisition unit is used to obtain result data corresponding to the processing result, and the result data is the complement of the processing result.
[0169] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.
[0170] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, computer software, or a combination of both.
[0171] In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0172] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A weight compression method, comprising: Obtaining a first code corresponding to an initial weight of a neural network model, where the first code is an original code corresponding to the initial weight; quantizing the first code into a second code, wherein the second code includes at least one target value; The second code is bit compressed to obtain a target weight, wherein the values of each group of valid position sequences in the target weight correspond to the positions of each target value in the second code.
2. According to the method of claim 1, the first bit of each bit of the second code is a sign bit, and the remaining bits are value bits; the value bits include at least one target value, and the value of the second code corresponds to the value of the initial weight; The step of compressing the second code by bits to obtain a target weight includes: The target weight is formed based on the sign bit and the bit position of the target value in the value bit of the second code; wherein the first bit of the target weight is the sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and each group of valid position sequences is a binary number corresponding to the position number of the bit position of each target value in the second code.
3. A data processing method, comprising: Obtaining a target weight of the neural network model, wherein the target weight is obtained by quantizing and compressing the original code corresponding to the initial weight in the neural network model; Obtaining a third code corresponding to the data to be processed, wherein the third code is an original code corresponding to the data to be processed; The third code is at least shifted based on the target weight to obtain a processing result corresponding to the data to be processed.
4. The method according to claim 3, wherein the target weight comprises a numerical sequence; wherein, The first bit of the numerical sequence is a sign bit, which is used to represent the sign bit of the original code corresponding to the initial weight; the other bits are numerical bits, and the numerical bits include multiple groups of valid position sequences corresponding to the multiple target values and arranged in order, and the numerical value of each group of valid position sequences represents the bit position of the corresponding target value in the first code.
5. The method according to claim 3, wherein the at least shifting processing is performed on the third code based on the target weight to obtain the processing result corresponding to the data to be processed, comprising: In a case where both the target weight and the third code are not zero, determining each shift number of the third code based on the target weight, wherein the shift number is a value of each group of valid position sequences in the target weight; Based on each of the shift numbers, the numerical bits in the third code are shifted and then added to obtain a shift result; Calculating the value of the sign bit of the target weight and the value of the sign bit of the third code to obtain a sign value; Based on the symbol value and the shift result, a processing result corresponding to the data to be processed is generated.
6. The method according to claim 5, wherein the step of calculating the value of the sign bit of the target weight and the value of the sign bit of the third encoding to obtain the sign value comprises: An XOR process is performed on the value of the sign bit of the target weight and the value of the sign bit of the third code to obtain a sign value.
7. The method according to claim 3, wherein obtaining the third code corresponding to the data to be processed comprises: When the operation condition of the target network layer of the neural network model meets the first condition, obtaining a third code corresponding to the data to be processed; The first condition indicates that the target network layer supports native code operation.
8. The method according to claim 7, after obtaining the processing result corresponding to the data to be processed, further comprises: Result data corresponding to the processing result is obtained, where the result data is the complement of the processing result.
9. A weight compression device, comprising: A first acquisition unit, used to acquire a first code corresponding to an initial weight of a neural network model, wherein the first code is an original code corresponding to the initial weight; A quantization unit, configured to quantize the first code into a second code, wherein the second code includes at least one bit as a target value; A compression unit is used to perform bit compression on the second code to obtain a target weight, wherein the values of each group of valid position sequences in the target weight correspond to the positions of each target value in the second code.
10. A data processing device, comprising: A second acquisition unit is used to obtain a target weight of the neural network model, wherein the target weight is obtained by quantizing and compressing the original code corresponding to the initial weight in the neural network model; A third acquisition unit, used to obtain a third code corresponding to the data to be processed, wherein the third code is an original code corresponding to the data to be processed; A processing unit is used to perform at least a shift process on the third code based on the target weight to obtain a processing result corresponding to the data to be processed.