Data processing method and device
By quantizing and compressing the target weights of the neural network model and compensating and shifting the data to be processed, the problem of slowing down the neural network when processing negative characteristic data is solved, and data processing efficiency is improved.
Patent Information
- Application Number
- CN202311362393.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-19
- Publication Date
- 2025-04-22
AI Technical Summary
When processing feature data, mainstream neural networks need to frequently process negative sign bits, resulting in slowing down processing speed.
By obtaining the target weight of the neural network model, quantizing and bit-digit compression, applying the accumulated compensation value to compensate the data to be processed, turning it into a positive value, and then shifting the data to be processed based on the target weight to obtain the processing result.
It improves the speed of neural network models when processing data, avoids frequent symbol bit processing, and simplifies the data processing process.
Smart Images
Figure CN119861897A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and device. Background Art
[0002] The feature data of mainstream neural networks are generally signed numbers, which are represented in the form of two's complement. The multiplication of the two's complement form requires first expanding the sign bit of the multiplicand, and then dividing it into two parts, the value bit and the sign bit, for multiplication. The value bit can be optimized to a shift operation through the above quantization, but the sign bit still needs to use the original complex inversion plus 1, and then perform the 'AND' operation.
[0003] In the middle layer of the network, after certain operators are applied, the feature data may become negative. In the convolutional layer or the fully connected layer, it is usually necessary to operate on the sign bit of the feature data that represents a negative number. Each layer of the neural network requires multiple operations. If the sign bit needs to be processed for each operation, it will affect the processing speed of the neural network. Summary of the invention
[0004] In view of this, the present application provides a data processing method, comprising:
[0005] Obtaining a target weight of a neural network model, wherein the target weight is obtained by quantizing an initial weight of the neural network model into a first code and compressing the number of bits of the first code; the first code includes a number of target values that is less than a preset value, and the target weight is used to characterize a bit position of each target value in the first code;
[0006] Applying the accumulated compensation value to perform compensation processing on the initial data to be processed, so that the initial data to be processed is converted into positive value data to be processed;
[0007] The data to be processed is first processed based on the target weight to obtain a first processing result corresponding to the data to be processed.
[0008] In the above method, the first bit of each bit of the first code is a sign bit, and the remaining bits are numerical bits; the number of target numerical values in the numerical bits is less than a preset value, and the numerical value of the first code corresponds to the numerical value of the initial weight.
[0009] The above method, the process of compressing the bit number of the first code to obtain the target weight, includes:
[0010] If the value bits of the first code do not contain the target value, using the value of the sign bit of the first code as the target weight;
[0011] If the numerical bits of the first code contain multiple target numerical values, the target weight is generated based on the bit positions of the target numerical values in the first code; wherein the first bit of the target weight is a sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and the numerical bits include multiple groups of valid position sequences corresponding to the multiple target numerical values and arranged in order, and the numerical value of each group of valid position sequences represents the bit position of the corresponding target numerical value in the first code.
[0012] In the above method, the first bit of the initial data to be processed is a sign bit, and the compensation value is greater than or equal to the maximum value allowed by the data type to which the initial data to be processed belongs;
[0013] The applying the accumulated compensation value to perform compensation processing on the initial data to be processed includes:
[0014] Only when the value of the initial data to be processed is less than zero, the accumulated compensation value is applied to perform compensation processing on the initial data to be processed.
[0015] In the above method, the first processing of the data to be processed based on the target weight to obtain a first processing result corresponding to the data to be processed includes:
[0016] In the case where both the target weight and the data to be processed are not zero, determining at least one shift number of the data to be processed based on the target weight; wherein the value of each group of valid position sequences in the target weight corresponds to the number of shifts each time;
[0017] Shifting the data to be processed respectively according to a plurality of shift numbers and then accumulating them to obtain a shift result;
[0018] Based on the sign bit of the target weight and the shift result, a first processing result corresponding to the data to be processed is obtained.
[0019] The above method further includes:
[0020] Obtaining a bias constant, wherein the bias constant at least includes a calculation result of the accumulated compensation value after the first processing and an initial bias value of the network;
[0021] Based on the first processing result and the bias constant, a second processing result corresponding to the initial data to be processed is obtained.
[0022] A data processing device, comprising:
[0023] An acquisition unit is used to acquire a target weight of a neural network model, wherein the target weight is obtained by quantizing an initial weight of the neural network model into a first code and compressing the number of bits of the first code; the first code includes a number of target values that is less than a preset value, and the target weight is used to characterize a bit position of each target value in the first code;
[0024] A first processing unit, configured to apply the accumulated compensation value to perform compensation processing on the initial data to be processed, so that the initial data to be processed is converted into positive data to be processed;
[0025] The second processing unit is used to perform a first processing on the data to be processed based on the target weight to obtain a first processing result corresponding to the data to be processed.
[0026] In the above device, the second processing unit includes:
[0027] At least one shift unit, corresponding to the shift number corresponding to each group of valid position sequences in the target weight, each of the shift units shifts the data to be processed according to its corresponding shift number, and outputs a shift value; wherein the first bit of the target weight is a sign bit, and the other bits are value bits, each value bit includes multiple groups of valid position sequences, and the number of shift units is consistent with the number of groups of valid position sequences;
[0028] A first accumulating unit, used for accumulating the shift values output by each shifting unit;
[0029] The result unit is used to combine the accumulation result of the accumulation unit and the sign bit of the target weight and then output them.
[0030] In the above device, the second processing unit further includes:
[0031] The analyzing unit is used to analyze the target weight, obtain each shift number, and input each shift number to each shift unit respectively.
[0032] The above device further comprises:
[0033] A constant unit, used to provide a bias constant;
[0034] The second accumulation unit is used to accumulate the result output by the result unit and the bias constant and then output the accumulated result. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0036] Figure 1 A method flow chart of a data processing method provided in an embodiment of the present application;
[0037] Figure 2 A flowchart of another method of a data processing method provided in an embodiment of the present application;
[0038] Figure 3 A device structure diagram of a data processing device provided in an embodiment of the present application;
[0039] Figure 4 Another device structure diagram of a data processing device provided in an embodiment of the present application;
[0040] Figure 5 A diagram of another device structure of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0041] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0042] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0043] The present application can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, etc.
[0044] The present application embodiment provides a data processing method, which can be applied to a variety of system platforms, and its execution subject can be a processor of a computer terminal or various mobile devices. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:
[0045] S101: Obtain target weights of a neural network model.
[0046] Among them, the target weight is obtained by quantizing the initial weight of the neural network model into a first code and compressing the bit number of the first code; the first code includes a number of target values that is less than a preset value, and the target weight is used to characterize the bit position of each target value in the first code.
[0047] Specifically, after the training of the neural network model is completed, the initial weights in the neural network model are quantized to obtain the first code. The target value in the first code is "1", and the number of target values in the first code is less than the preset value, which is the number of bits of the first code. The initial weights can be other int type data such as int8, int16 or int32, so after the initial weights are quantized to obtain the first code, the first code is also int data. Taking int8 data as an example, since the value range of int8 data is -128 to 127, that is, -2 7 ~2 7 -1, the preset value ranges from 0 to 7. For int8 data, in order to facilitate the subsequent processing of the data to be processed, the preset value can be set to 2 so that the first code obtained after quantizing the initial weight contains at most two target values.
[0048] Among them, the first bit of each bit of the first code is a sign bit, and the remaining bits are value bits. The value of the sign bit can be "0" or "1" to represent the sign of the first code, wherein the value of the sign bit is "0" to represent the sign "+", and the value of the sign bit is "1" to represent the sign "-". The value of the first code corresponds to the value of the initial weight.
[0049] It should also be noted that the initial weight is an int8 binary number. After the initial weight is quantized into a first code, the value of the first code is the sum of a finite number of powers of 2, where the number of powers of 2 corresponding to the first code is less than the preset value.
[0050] Taking int8 as an example, if the initial weight is 53=(00110101), the quantization is (00110000)=48; if the initial weight is 67=(01000011), the quantization is (01000100)=68; if the initial weight is 63=(00111111), the quantization is (01000000)=64.
[0051] Optionally, after the initial weight is quantized, the first code obtained may include multiple sequence groups, each sequence group is a binary number, and the value of each binary number is consistent with the value of the initial weight. For example, the initial weight 48=00110000 can be quantized into a first code including two sequence groups, and the first code is 48=2 5 +2 4 =(00100000)+(00010000), where the first sequence group is 00100000 and the second sequence group is 00010000.
[0052] In the present application, after obtaining the first code, the first code is compressed to obtain the target weight of the neural network model. The process of compressing the first code is as follows:
[0053] If the value bits of the first code do not contain the target value, the value of the sign bits of the first code is used as the target weight;
[0054] If the numerical bits of the first code contain multiple target numerical values, a target weight is generated based on the bit position of the target numerical value in the first code; wherein the first bit of the target weight is a sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and the numerical bits include multiple groups of valid position sequences corresponding to the multiple target numerical values and arranged in order, and the numerical value of each group of valid position sequences represents the bit position of the corresponding target numerical value in the first code.
[0055] It should be noted that the initial weights of the neural network model are binary data of the int type, and each group of valid position sequences in the target weights obtained after quantizing and compressing the initial weights are also binary data.
[0056] For example, the first code is 80=01010000, with the rightmost bit position being the 0th bit, and counting from right to left, the bit positions of the target value "1" are the 6th and 4th bits respectively, and the binary numbers corresponding to the two bit positions are 4=100 and 6=110; since the sign bit in the first code is "0", the target weight finally obtained according to the order of the bit positions of the target values is: 0110100, where one set of valid position sequences of the target weight is 110, and the other set of valid position sequences is 100.
[0057] The target weights can be obtained by processing the initial weights provided by the model trainer and configuring them in the model when the network is deployed; or by converting the initial weights each time the model is used to continue data processing. If the target weights are configured in the model after the initial weights are quantized and compressed when the network is deployed, then when the neural network model is used for data processing, the target weights are directly applied to implement the data processing process of the neural network model. If the target weights are obtained by quantizing and compressing the initial weights each time the neural network model is used, then before the neural network model is used for data processing, the initial weights in the neural network model are first quantized and compressed, and after obtaining the target weights, the target weights are applied to implement the data processing process of the neural network model.
[0058] S102: Apply the accumulated compensation value to perform compensation processing on the initial data to be processed, so that the initial data to be processed is converted into positive value data to be processed.
[0059] The initial data to be processed is the feature data after being input into the neural network model, or the feature data after being input into the neural network model and preprocessed. The data to be processed can specifically be text, image, video or audio data.
[0060] The first bit of the initial data to be processed is a sign bit, and the compensation value is greater than or equal to the maximum value allowed by the data type to which the initial data to be processed belongs. Therefore, the specific method of applying the accumulated compensation value to compensate the initial data to be processed is: only when the value of the initial data to be processed is less than zero, the accumulated compensation value is applied to compensate the initial data to be processed.
[0061] It should be noted that the cumulative compensation value is set according to the data type of the data to be processed. If the data to be processed is int8 data, since its corresponding value range is -128 to 127, the cumulative compensation value is set to 128. Regardless of the value of the initial data to be processed, the value obtained by adding the cumulative compensation value is a positive number.
[0062] For example, if the data to be processed is -87=11010111, the data to be processed obtained after applying the accumulated compensation value 128 to compensate the data to be processed is -87+128=41=00101001.
[0063] S103: Performing a first process on the data to be processed based on the target weight to obtain a first processing result corresponding to the data to be processed.
[0064] It should be noted that the first processing refers to shift processing of the data to be processed, specifically, shift processing of the numerical bits in the data to be processed. The first bit of the data to be processed is a sign bit, and the other bits are numerical bits.
[0065] Since the data to be processed is a binary number obtained after the initial data to be processed is compensated, when the target weight is applied to perform the first processing on the data to be processed, the result obtained is the processing result of the compensated data, rather than the processing result of the initial data to be processed.
[0066] Among them, the shift processing of the data to be processed is to determine multiple shift numbers for the shift processing of the data to be processed according to the target weight, and add the data to be processed after shift processing by each shift number, so as to obtain the first processing result of the data to be processed. Therefore, it is also necessary to perform a first processing on the accumulated compensation value based on the target weight to obtain the compensation processing result. The difference between the first processing result and the compensation processing result is calculated to obtain the final processing result of the initial data to be processed.
[0067] Since the value of the data to be processed is positive, after obtaining a processing result corresponding to the data to be processed, the value of the sign bit of the first processing result is consistent with the value of the sign bit in the target weight. The present application only needs to shift the data to be processed to obtain the final result, while the prior art requires multiple multiplication and addition operations to obtain the final result. Taking the initial data to be processed as -85 and the initial weight as 87 as an example, the binary complement of the initial data to be processed is: 10101011, and the binary complement of the initial weight is: 01010111. If the initial data to be processed needs to be multiplied by the initial weight, the calculation process of the multiplication and addition algorithm of the prior art includes numerical bit operations and sign bit operations, wherein the numerical bit operation process is as follows:
[0068] 1111111110101011+1111111101010110+11111110101100+0000000000000000+1111101010110000+0000000000000000+1110101011000000=1110001100011101
[0069] The sign bit operation process is as follows:
[0070] {(~{110101011}+1)&0,7'b0}=0000000000000000
[0071] Therefore, the calculation result of the prior art is:
[0072] 1110001100011101+0000000000000000=1110001100011101
[0073] Among them, since the binary number obtained by multiplying two numbers exceeds the original number of bits, the multiplicand -85 = (10101011) is first calculated before the multiplication operation. 补 Expanded to 11111111110101011. It can be seen from the above formula that the calculation process of the multiplication using the prior art requires multiple addition operations. The larger the initial weight, the longer the required addition process. Using the method provided in embodiments S101 to S103 of the present application, the initial weight 87=01010111 is quantized to obtain the first code 80=01010000, and then the first code is compressed to obtain a target weight of 0110100, wherein the target weight includes two groups of valid position sequences, the first group of valid position sequences is: 110, and the second group of valid position sequences is: 100; according to the two groups of valid position sequences, the shift numbers for the shift processing are determined to be 6 and 4 respectively; since the data to be processed is a negative value, the data to be processed is compensated with the accumulated compensation value (128), and the data to be processed is: -85+128=43=(00101011) 补 Therefore, the process of shifting the numerical bits of the data to be processed based on the target weight is as follows:
[0074] 101011<<6=101011000000
[0075] 101011<<4=1010110000
[0076] 101011000000+1010110000=110101110000=3440
[0077] Since the data to be processed is a positive value, the sign bit value of the first processing result obtained is consistent with the sign bit value of the target weight, and the first processing result corresponding to the data to be processed is: 3440 = (0110101110000) 补 , where the first "0" represents the symbol "+". Since the accumulated compensation value and the target weight are already preset, the neural network model has determined the compensation result of the first processing of the accumulated compensation value according to the target weight before processing the initial data to be processed: 10240 = (010100000000000) 原 , since the compensation processing result is greater than the first processing result, 3400-10240 can be converted to 3400+(-10240), and the complement of the value -10240 is: (1001100000000000) 补 , where the first bit in the complement code is the sign bit, so the final processing result corresponding to the initial data to be processed obtained according to the first processing result and the compensation processing result is:
[0078] 110101110000+1100000000000=10010101110000
[0079] The binary number obtained based on the above formula is in the form of a complement code. Therefore, the final processing result corresponding to the initial data to be processed is -6800 = (110010101110000) 补 , where the first bit value "1" indicates that the processing result is a negative value, and the other bits are numerical bits.
[0080] It should be noted that, in the process of quantizing the initial weight into the first code, it is necessary to consider that the value of the quantized first code is close to the value of the initial weight. After compressing the first code, the result obtained by calculation is close to the result of applying the initial weight to the processed data and calculating in the prior art, so after quantizing and compressing the initial weight, the accuracy of the neural network model can still be maintained.
[0081] It can be seen from the above examples that in the process of applying the prior art to perform multiplication calculations on the initial weight and the data to be processed, a large number of additions are required to obtain the final result, and in the calculation process, it is also necessary to calculate according to the sign bit of the data to be processed and the sign bit of the initial weight, which makes the data processing process too complicated. The data processing method of the present application only needs to set the target weight and the compensation processing result corresponding to the accumulated compensation value in advance before data processing. When data processing is required, if the value of the initial data to be processed is greater than zero, the initial data to be processed is directly shifted based on the target weight to obtain the final processing result, without the need for symbolic operation or a large number of addition operations; if the initial data to be processed is less than zero, the initial data to be processed is processed based on the accumulated compensation value to obtain the data to be processed with a positive value, and the first processing result is obtained by performing a first processing (shift processing) on the data to be processed based on the target weight, and then the compensation processing result that has been set is calculated with the first processing result to obtain the final processing result corresponding to the initial data to be processed.
[0082] The method provided in the embodiment of the present application replaces a large number of addition operations in the prior art with shift operations and converts the data into positive values, which can avoid the operation of signs when processing data, thereby improving the data processing speed in the neural network model.
[0083] In the method provided in the embodiment of the present application, the neural network model includes multiple network layers for data processing. In the process of processing data, some network layers usually add bias values to the results output after data processing, so the result output after the network layer finally processes the data satisfies the following formula:
[0084] f(x)=Wx+b (1)
[0085] Among them, w is the weight, b is the bias, and x is the data to be processed.
[0086] When no bias value is added to the network layer, the data processing process of the neural network model satisfies the following formula:
[0087] f(x)=Wx (2)
[0088] In the method provided in the embodiment of the present application, if the data to be processed is less than zero, the initial data to be processed x can be converted into the data to be processed y by referring to the following formula:
[0089] x=y-128 (3)
[0090] Combining the above formula (3), we can get:
[0091] f(x)=f(y-128) (4)
[0092] Since linear transformation is satisfied, formula (4) can be transformed into:
[0093] f(x)=f(y)-f(128) (5)
[0094] If the neural network model needs to add a bias value during data processing, then combining the above formulas (1) and (5), the formula for processing the data to be processed is:
[0095] f(x)=f(y-128)=W(y-128)+b=Wy-128W+b (6)
[0096] By integrating the constant b-128W into the constant B, the above formula (6) is converted to:
[0097] f(x)=Wy-128W+b=Wy+B (7)
[0098] If the neural network model does not need to add bias when processing data, then the result is equations (2) and (5), and the formula for processing the data to be processed is:
[0099] f(x)=f(y-128)=W(y-128)=Wy-128W (8)
[0100] It can be seen from the above formulas (7) and (8) that after completing the training of the neural network model, the initial weights in the neural network model are first quantized and compressed into target weights. If the neural network model needs to add a bias for data processing, a bias constant B is pre-set based on the above formula (7). The bias constant is obtained by adding the calculation result of the first processing of the cumulative compensation value 128 and the network initial bias value b of the neural network model. If the neural network model does not need a bias value for data processing, another bias constant -128W is preset based on the above formula (8). The bias constant represents the result obtained after the first processing of the cumulative compensation value 128.
[0101] Based on the method provided in the above embodiment, the way to obtain the processing result corresponding to the initial data to be processed is: obtain a bias constant; based on the first processing result and the bias constant, obtain a second processing result corresponding to the initial data to be processed. Wherein, the bias constant at least includes the calculation result of the cumulative compensation value after the first processing and the initial bias value of the network, or includes the calculation result of the cumulative compensation value after the first processing. In this application, the first processing result refers to obtaining each shift number based on the target weight, and accumulating the data after shifting according to multiple shift numbers.
[0102] Based on the method provided in the above embodiment, after the neural network model completes training, the target weights in the neural network model and the accumulated compensation values that need to be added during data processing are set in advance, and the bias constants required in the calculation process are obtained in advance based on the accumulated compensation values and the target weights. This ensures that after the feature data is input into the neural network model, and when the data to be processed is a negative value, it is only necessary to convert the data to be processed into a positive value and then perform the first processing according to the target weight and then accumulate the bias constant. There is no need to perform complex multiplication and addition operations and related symbol operations, thereby improving the efficiency of the neural network model in processing data.
[0103] The method provided in the embodiments of the present application is referred to as Figure 2 , performing a first processing on the data to be processed based on the target weight to obtain a first processing result corresponding to the data to be processed, including:
[0104] S201: When the target weight and the data to be processed are both not zero, determining at least one shift number of the data to be processed based on the target weight.
[0105] Among them, the value of each group of valid position sequences in the target weight corresponds to the number of shifts each time.
[0106] It is understandable that if either the target weight or the data to be processed is zero, the product of the two is zero, and further calculation is not required. If both the target weight and the data to be processed are not zero, each shift number is determined according to the target weight.
[0107] In this application, the target weight is a binary number obtained after the initial weight is quantized and compressed. The quantization and compression of the weight is to quantize the weight to be represented by a finite power of 2, and represent the quantized result in a compressed form. The weight can be quantized to int8, and then int8 can be quantized to a finite power of 2 (2 n ). In the following description, the weight is quantized into two powers of 2 as an example. The value range of int8 is -128 to 127, that is, -2 7 ~2 7 -1, limits the weight value range to this range, and the specific quantized form is:
[0108] ±(2 b0 +2 b1 ), b0,b1∈{0,1,2,3,4,5,6,7}(9)
[0109] Because the power exponent ranges from 0 to 7, it can be represented by 3 bits. The highest bit of the quantized weight represents the sign bit, and 3 bits represent the power exponent. When the weight is quantized to multiple powers of 2, multiple 3 bits are used to represent it. When quantized to two powers of 2, the weight can be represented by 7 bits. For example, when the weight is -68 = -(2 6 +2 2 ), the 7-bit target weight is represented as 1110010, and the first digit 1 represents a negative number; when the weight is -48 = -(2 5 +2 4 ), the 7-bit target weight is represented as 1101100; when the weight is 96 = 2 6 +2 5 , the 7-bit target weight is represented as 0110101. In particular, when the weight is 0, it is represented by 0111.
[0110] It can be understood that, among the bits of the target weight, except for the first bit which is a sign bit, the other bits are value bits, and every 3 bits of value bits correspond to a set of valid position sequences.
[0111] S202: Shift the data to be processed respectively according to a plurality of shift amounts and then accumulate them to obtain a shift result.
[0112] Specifically, the first processing is to shift the data to be processed according to each shift number. For example, the data to be processed is a, and the sign bit of the target weight is sign b , the value is 2 b0 +2 b1 , then the calculation formula corresponding to the first processing is:
[0113] c=a*(2 b0 +2 b1)=(a<<b0)+(a<<b1) (10)
[0114] S203: Obtain a first processing result corresponding to the data to be processed based on the sign bit of the target weight and the shift result.
[0115] Since only the target weight can be negative, the sign bit of the first processing result is consistent with the sign bit of the target weight:
[0116] sign c =sign b (11)
[0117] According to the results obtained from the above equations (10) and (11), the first processing result is {sign c , c}.
[0118] In the method provided in the embodiment of the present application, by quantizing and compressing the initial weights of the neural network model, the multiplication and addition operations of data and weights in the prior art can be converted into shift processing. Data can be processed more quickly without the need for a large number of multiplication and addition operations, thereby improving the efficiency of data processing in the neural network model.
[0119] The specific implementation processes and derivative methods of the above-mentioned embodiments are all within the protection scope of this application.
[0120] and Figure 1 Corresponding to the method described above, the embodiment of the present application further provides a data processing device for Figure 1 In the specific implementation of the method, the data processing device provided in the embodiment of the present application can be applied to a computer terminal or various mobile devices, and its structural diagram is as follows Figure 3 As shown, specifically including:
[0121] An acquisition unit 301 is used to acquire a target weight of a neural network model, wherein the target weight is obtained by quantizing an initial weight of the neural network model into a first code and compressing the number of bits of the first code; the first code includes a number of target values that is less than a preset value, and the target weight is used to characterize a bit position of each target value in the first code;
[0122] The first processing unit 302 is used to apply the accumulated compensation value to perform compensation processing on the initial data to be processed, so that the initial data to be processed is converted into positive data to be processed;
[0123] The second processing unit 303 is used to perform a first processing on the data to be processed based on the target weight to obtain a first processing result corresponding to the data to be processed.
[0124] refer to Figure 4 In this embodiment of the present invention, the second processing unit 303 includes:
[0125] At least one shift unit 401, used to correspond to the shift number corresponding to each group of valid position sequences in the target weight, each of the shift units 401 shifts the data to be processed according to its corresponding shift number, and outputs a shift value; wherein the first bit of the target weight is a sign bit, and the other bits are value bits, each value bit includes multiple groups of valid position sequences, and the number of shift units 401 is consistent with the number of groups of valid position sequences;
[0126] A first accumulating unit 402, configured to accumulate the shift values output by each shifting unit 401;
[0127] The result unit 403 is used to combine the accumulation result of the accumulation unit 402 and the sign bit of the target weight and output them.
[0128] In the device provided by the embodiment of the present invention, the second processing unit 303 further includes:
[0129] The analysis unit is used to analyze the target weight, obtain each shift number, and input each shift number to each shift unit 401 respectively.
[0130] refer to Figure 5 , the device provided in the embodiment of the present invention further includes:
[0131] A constant unit 404, used to provide a bias constant;
[0132] The second accumulation unit 405 is used to accumulate the result output by the result unit 403 and the bias constant and then output the accumulated result.
[0133] The specific working process of each unit and sub-unit in the data processing device disclosed in the above embodiment of the present application can be found in the corresponding content of the data processing method disclosed in the above embodiment of the present application, and will not be repeated here.
[0134] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.
[0135] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, computer software, or a combination of both.
[0136] In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0137] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A data processing method, comprising: Obtaining a target weight of a neural network model, wherein the target weight is obtained by quantizing an initial weight of the neural network model into a first code and compressing the number of bits of the first code; the first code includes a number of target values that is less than a preset value, and the target weight is used to characterize a bit position of each target value in the first code; Applying the accumulated compensation value to perform compensation processing on the initial data to be processed, so that the initial data to be processed is converted into positive value data to be processed; The data to be processed is first processed based on the target weight to obtain a first processing result corresponding to the data to be processed.
2. According to the method described in claim 1, the first bit of each bit of the first code is a sign bit, and the remaining bits are numerical bits; the number of target numerical values in the numerical bits is less than a preset value, and the numerical value of the first code corresponds to the numerical value of the initial weight.
3. According to the method of claim 2, the process of compressing the number of bits of the first code to obtain the target weight comprises: If the value bits of the first code do not contain the target value, using the value of the sign bit of the first code as the target weight; If the numerical bits of the first code contain multiple target numerical values, the target weight is generated based on the bit positions of the target numerical values in the first code; wherein the first bit of the target weight is a sign bit, the sign bit of the target weight is consistent with the sign bit in the initial weight, and the numerical bits include multiple groups of valid position sequences corresponding to the multiple target numerical values and arranged in order, and the numerical value of each group of valid position sequences represents the bit position of the corresponding target numerical value in the first code.
4. The method according to claim 1, wherein the first bit of the initial data to be processed is a sign bit, and the compensation value is greater than or equal to the maximum value allowed by the data type to which the initial data to be processed belongs; The applying the accumulated compensation value to perform compensation processing on the initial data to be processed includes: Only when the value of the initial data to be processed is less than zero, the accumulated compensation value is applied to perform compensation processing on the initial data to be processed.
5. According to the method of claim 3, performing a first processing on the data to be processed based on the target weight to obtain a first processing result corresponding to the data to be processed comprises: In the case where both the target weight and the data to be processed are not zero, determining at least one shift number of the data to be processed based on the target weight; wherein the value of each group of valid position sequences in the target weight corresponds to the number of shifts each time; Shifting the data to be processed respectively according to a plurality of shift numbers and then accumulating them to obtain a shift result; Based on the sign bit of the target weight and the shift result, a first processing result corresponding to the data to be processed is obtained.
6. The method according to claim 1 or 5, further comprising: Obtaining a bias constant, wherein the bias constant at least includes a calculation result of the accumulated compensation value after the first processing and an initial bias value of the network; Based on the first processing result and the bias constant, a second processing result corresponding to the initial data to be processed is obtained.
7. A data processing device, comprising: An acquisition unit is used to acquire a target weight of a neural network model, wherein the target weight is obtained by quantizing an initial weight of the neural network model into a first code and compressing the number of bits of the first code; the first code includes a number of target values that is less than a preset value, and the target weight is used to characterize a bit position of each target value in the first code; A first processing unit, configured to apply the accumulated compensation value to perform compensation processing on the initial data to be processed, so that the initial data to be processed is converted into positive data to be processed; The second processing unit is used to perform a first processing on the data to be processed based on the target weight to obtain a first processing result corresponding to the data to be processed.
8. The device according to claim 7, wherein the second processing unit comprises: At least one shift unit, corresponding to the shift number corresponding to each group of valid position sequences in the target weight, each of the shift units shifts the data to be processed according to its corresponding shift number, and outputs a shift value; wherein the first bit of the target weight is a sign bit, and the other bits are value bits, each value bit includes multiple groups of valid position sequences, and the number of shift units is consistent with the number of groups of valid position sequences; A first accumulating unit, used for accumulating the shift values output by each shifting unit; The result unit is used to combine the accumulation result of the accumulation unit and the sign bit of the target weight and then output them.
9. The device according to claim 8, wherein the second processing unit further comprises: The analyzing unit is used to analyze the target weight, obtain each shift number, and input each shift number to each shift unit respectively.
10. The apparatus according to claim 8, further comprising: A constant unit, used to provide a bias constant; The second accumulation unit is used to accumulate the result output by the result unit and the bias constant and then output the accumulated result.