Data processing method and device
By splitting the target shift number and performing multi-level shift operations, the problems of complex multiplication operations and high resource consumption in the neural network are solved, and the optimization of hardware resources and the improvement of model efficiency are achieved.
Patent Information
- Application Number
- CN202311362392.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-19
- Publication Date
- 2025-04-22
AI Technical Summary
Multiplication operations in neural networks are complex and consume a lot of hardware resources, which increases the cost in model training or inference.
By obtaining the target weight, the target shift number is split into common shift number and independent shift number, and the multiplication operation is simplified by multi-stage shift operations.
It reduces the consumption of hardware resources, simplifies the multiplication operation process, and improves the training and inference efficiency of the model.
Smart Images
Figure CN119861896A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and device. Background Art
[0002] Multiplication operation is the most important operation in neural networks, but the execution process of multiplication operation is relatively complicated, and a large number of operation circuits need to be deployed in the specific execution process. Therefore, the multiplication operation in neural networks currently consumes most of the resources of the hardware system, increasing the cost of model training or inference process. Summary of the invention
[0003] In view of this, the present application provides a data processing method, comprising:
[0004] Obtaining data to be processed and a target weight, wherein the target weight includes multiple groups of valid numerical sequences, and the values of each group of valid numerical sequences are the power values of each power of 2 after the initial weight is quantized into the sum of multiple powers of 2;
[0005] Based on the target weight, at least one target shift number is obtained, each of the target shift numbers being a value of all valid numerical value sequences or a value of a part of valid numerical value sequences in the target weight;
[0006] Executing the first strategy, and obtaining a processing result of the data to be processed based on the obtained second shift result;
[0007] The first strategy includes:
[0008] Based on the maximum shift number of a single shift operation supported by the hardware, split each of the target shift numbers into the same common shift number and respective independent shift numbers;
[0009] The data to be processed are respectively shifted by one level by a plurality of independent shift numbers and then added to obtain a first shift result;
[0010] The first shift result is subjected to secondary shifting according to the common shift number to obtain a second shift result.
[0011] In the above method, the first bit of the target weight is a sign bit, and the other bits are value bits, and each of the value bits includes multiple groups of valid value sequences.
[0012] In the above method, obtaining the target shift number based on the target weight includes:
[0013] Based on the target weight, a plurality of initial shift numbers are obtained, each of which is a value of all valid value sequences in the target weight;
[0014] Determining the difference between each of the initial shift numbers;
[0015] If the differences between the initial shift numbers are all smaller than the first preset difference, the initial shift numbers are the target shift numbers;
[0016] If the difference between the initial shift numbers is not less than the first preset difference, the initial shift numbers are divided into multiple groups of target shift numbers, and the difference between the target shift numbers in each group of target shift numbers is less than the first preset difference.
[0017] In the above method, executing the first strategy includes:
[0018] If each of the initial shift numbers is divided into multiple groups of target shift numbers, multiple rounds of the first strategy are executed.
[0019] In the above method, obtaining the processing result of the data to be processed based on the obtained second shift result includes:
[0020] The second shift results obtained by executing the first strategy each time are added together to obtain the processing result of the data to be processed.
[0021] In the above method, the common shift number is less than or equal to the secondary shift number supported by the hardware.
[0022] In the above method, the common shift number is the minimum shift number among the target shift numbers.
[0023] In the above method, each of the independent shift numbers is not greater than the maximum shift number supported by the first-level shift, and the common shift number is not greater than the maximum shift number supported by the second-level shift.
[0024] A data processing device, comprising:
[0025] A first acquisition unit is used to acquire the data to be processed and the target weight, wherein the target weight includes multiple groups of valid numerical sequences, and the values of each group of valid numerical sequences are the power values of each power of 2 after the initial weight is quantized into the sum of multiple powers of 2;
[0026] A second acquisition unit is used to obtain at least one target shift number based on the target weight, each of which is a value of all valid numerical value sequences or a value of part of the valid numerical value sequences in the target weight;
[0027] A processing unit, configured to execute the first strategy and obtain a processing result of the data to be processed based on the obtained second shift result;
[0028] The first strategy includes:
[0029] Based on the maximum shift number of a single shift operation supported by the hardware, split each of the target shift numbers into the same common shift number and respective independent shift numbers;
[0030] The data to be processed are respectively shifted by one level by a plurality of independent shift numbers and then added to obtain a first shift result;
[0031] The first shift result is subjected to secondary shifting according to the common shift number to obtain a second shift result.
[0032] In the above device, the processing unit comprises:
[0033] A plurality of first-stage shift units, each first-stage shift unit corresponds to each independent shift number one by one, and is used to perform first-stage shift on the data to be processed according to its corresponding independent shift number;
[0034] An adding unit, used for adding the results after the first-stage shift according to the common shift number, and outputting a first shift result;
[0035] A secondary shift unit, used for performing secondary shift on the first shift result and outputting a second shift result;
[0036] A result unit is used to output a processing result corresponding to the data to be processed according to the second shift result. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0038] Figure 1 A method flow chart of a data processing method provided in an embodiment of the present application;
[0039] Figure 2 A flowchart of another method of a data processing method provided in an embodiment of the present application;
[0040] Figure 3 A flowchart of another method of a data processing method provided in an embodiment of the present application;
[0041] Figure 4 A device structure diagram of a data processing device provided in an embodiment of the present application;
[0042] Figure 5 A schematic diagram of a circuit structure corresponding to a data processing device provided in an embodiment of the present application;
[0043] Figure 6 A schematic diagram of another circuit structure corresponding to a data processing device provided in an embodiment of the present application;
[0044] Figure 7 A schematic diagram of another circuit structure corresponding to a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0046] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprise one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0047] The present application can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, etc.
[0048] The present application embodiment provides a data processing method, which can be applied to a variety of system platforms, and its execution subject can be a processor of a computer terminal or various mobile devices. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:
[0049] S101: Obtain data to be processed and target weight.
[0050] The target weight includes multiple sets of valid numerical sequences, and the values of each set of valid numerical sequences are the power values of each power of 2 after the initial weight is quantized to the sum of multiple powers of 2. For example, if the initial weight is 48=00110000, the corresponding sum of multiple powers of 2 is 48=2 5 +24 , according to the power value 5 and the power value 4, the target weight obtained is 0101100, and the valid numerical sequences in the target weight are 101 and 100 respectively.
[0051] The data to be processed is the feature data after being input into the neural network model, or the feature data input into the neural network model and preprocessed. Specifically, the data to be processed may be text, image, video or audio data.
[0052] Each set of valid numerical sequences in the target weights is a binary data, and the data to be processed is the binary data corresponding to the feature data of the input neural network model.
[0053] In the present application, the first bit of each bit in the target weight is a sign bit, and the other bits are value bits, and each value bit includes multiple groups of valid value sequences.
[0054] The target weights can be obtained by processing the initial weights provided by the model trainer and configuring them in the model when the network is deployed; or by converting the initial weights each time the model is used to continue data processing. If the target weights are configured in the model after the initial weights are quantized and compressed when the network is deployed, then when the neural network model is used for data processing, the target weights are directly applied to implement the data processing process of the neural network model. If the target weights are obtained by quantizing and compressing the initial weights each time the neural network model is used, then before the neural network model is used for data processing, the initial weights in the neural network model are first quantized and compressed, and after obtaining the target weights, the target weights are applied to implement the data processing process of the neural network model.
[0055] S102: Obtain at least one target shift number based on the target weight.
[0056] Among them, each target shift number is the value of all valid numerical value sequences or the value of part of the valid numerical value sequences in the target weight.
[0057] It should be noted that in the process of applying the weights in the neural network model to process the data to be processed, the data to be processed can be processed once or multiple times. When the data to be processed needs to be processed once, the target shift numbers obtained are the values of all valid value sequences in the target weights; when the data to be processed needs to be processed multiple times, the values of each valid value sequence in the target weights are split into multiple value groups, and when the data to be processed is processed according to the number of split groups, the values of each valid value sequence in the current value group are the target shift numbers.
[0058] S103: Execute the first strategy, and obtain a processing result of the data to be processed based on the obtained second shift result.
[0059] It should be noted that the execution of the first strategy is mainly to shift the data to be processed separately according to each target shift data; since in the shifting process, there may be a large difference between multiple target shift numbers, resulting in the shift results obtained by using hardware to shift are not synchronized, so the first strategy is used to split each target shift number before shifting.
[0060] refer to Figure 2 , the first strategy includes the following execution steps:
[0061] S201: Based on the maximum shift number of a single shift operation supported by the hardware, split each target shift number into the same common shift number and respective independent shift numbers.
[0062] The maximum number of shifts supported by the hardware for a single shift operation is related to the number of bits in the initial weight. If the number of bits in the initial weight is 8 bits, the value range of the initial weight after quantization is -2 7 ~2 7 Therefore, the maximum number of shifts supported by the hardware for a single shift operation is no greater than 7, that is, the maximum shift range is [0,7].
[0063] In this application, if any of the target shift numbers is greater than the maximum shift number, it is necessary to split each target shift number into a common shift number and each independent shift number. For example, if the target shift numbers are 3, 4, and 5, and the maximum shift number of a single shift operation supported by the hardware is 4, then the common coefficient "3" can be extracted as the common shift number, and the independent shift numbers corresponding to each target shift number are 0, 1, and 2.
[0064] Among them, in the hardware, multi-level shifting can be performed on the processed data according to the target shift number. Taking the hardware supporting the second-level shift as an example, if the number of bits of the initial weight is 8 bits, then the sum of the maximum shift number supported by the first-level shift and the maximum shift number of the second-level shift shall not be less than the value 7. If the maximum shift number of the first-level shift is greater than the maximum shift number of the second-level shift, then the maximum shift number of the second-level shift is the maximum shift number of a single shift operation supported by the hardware; if the maximum shift number of the first-level shift is less than the maximum shift number of the second-level shift, then the maximum shift number of the first-level shift is the maximum shift number of a single shift operation supported by the hardware.
[0065] For example: the maximum shift number supported by the first-level shift is 4, and the maximum shift number supported by the second-level shift is 7. Therefore, the shift numbers of each first-level shift cannot exceed 4. If there are shift numbers greater than 4 among the target shift numbers, each target shift number needs to be split into a common shift number and its own independent shift number. After the split, each independent shift number obtained shall not be greater than 4, and the common shift number shall not be greater than 7.
[0066] Another example: the maximum shift number for the first-level shift is 4, and the maximum shift number for the second-level shift is 3. Therefore, the shift numbers for the first-level shift cannot exceed 4, and the shift numbers for the second-level shift cannot exceed 3. If there are shift numbers greater than 3 among the target shift numbers, then the target shift numbers need to be split into a common shift number and their own independent shift numbers. The independent shift numbers are no more than 4, and the common shift numbers are no more than 3.
[0067] Specifically, the method of splitting each target shift number into the same common number of bits and respective independent shift numbers includes at least one of the following:
[0068] (1) The smallest shift number among all target shift numbers is taken as the common shift number, and the difference between the target shift number and the common shift number is taken as the independent shift number.
[0069] (2) According to the secondary shift number supported by the hardware, any common value bit in each target shift number is extracted as the common shift number, and the difference between the target shift number and the common shift number is the independent shift number.
[0070] S202: Shift the data to be processed by one level respectively according to a plurality of independent shift amounts and then add them together to obtain a first shift result.
[0071] It should be noted that when performing a primary shift according to multiple independent shift numbers, only the numerical bits in the data to be processed are shifted. The data to be processed is a binary number, and in each bit of the binary number, the first bit is a sign bit, and the other bits are numerical bits.
[0072] For example, the data to be processed is 48=00110000, and the first value "0" in each bit indicates that the data to be processed is a positive value; if the independent shift numbers are 0, 1, and 2, respectively, the initial shift results obtained after shifting the data to be processed according to the independent shift numbers are 0000000000110000, 0000000001100000 and 00000000011000000, respectively, and the three initial shift results are added to obtain the first shift result of 0000000101010000.
[0073] It is understandable that before the data to be processed is shifted by one level, since the number of bits after the binary number is shifted will exceed the original number of bits, the data to be processed can be expanded in advance. Among them, the expansion of the data to be processed can be to add multiple values consistent with the value of the sign bit to the left side of the data to be processed. For example, if the value of the sign bit is "0", multiple bits with a value of "0" can be added to the left side of the data to be processed; if the value of the sign bit is "1", multiple bits with a value of "1" can be added to the left side of the data to be processed.
[0074] S203: Perform secondary shifting on the first shift result according to the common shift number to obtain a second shift result.
[0075] Based on the method provided in the above embodiment, the process of performing primary shift and secondary shift on the data to be processed includes the following embodiments:
[0076] Taking the example corresponding to the above S202 as an example, if the common shift number is 3, after obtaining the first shift result of 0000000101010000, the first shift result 0000000101010000 is shifted according to the common shift number to obtain a second shift result of 0000101010000000.
[0077] Optionally, if each target shift number is not greater than the maximum shift number of a single shift operation supported by the hardware, each target shift number may not be split, and only one-stage shift is performed on the data to be processed according to each target shift number. Alternatively, after splitting, the common shift number is 0, and each independent shift number is consistent with each target shift number.
[0078] In the embodiment of the present application, if each target shift number is the value of all valid numerical sequences in the target weight, the second shift result obtained after executing the first strategy is the final processing result of the data to be processed. If each target shift number is the value of a part of the valid numerical sequence in the target weight, then after executing the first strategy multiple times according to each target shift number corresponding to the value of the valid numerical sequence of each part, the second shift results obtained by each execution of the first strategy are added to obtain the processing result corresponding to the data to be processed.
[0079] In the embodiment of the present application, the data to be processed can be shifted according to multiple target shift numbers. For example, b is the target weight, b=2 m +2 n , then the target shift numbers are m and n, and the data to be processed is a. Then the final shift result can be obtained by adding the shifted data to be processed:
[0080] a*b=a*(2 m +2 n )=a*2 n +a*2 m =a<<n+a<<m (1)
[0081] Among them, the symbol “<<” in the above formula is a shift symbol.
[0082] Furthermore, if any of the target shift numbers m and n is greater than the maximum shift number of a single shift supported by the hardware, the common coefficients in each target shift number can be extracted as the common shift number, and the value after extracting the common coefficient is the independent shift number. For example: if n = k + i, m = k + j, then the above formula (1) can be transformed into:
[0083] a*b=a*(2 n +2 m )=a*(2 k+i +2 k+j ) (2)
[0084] According to formula (2), the shift formula can be obtained:
[0085] a*b=a<<(k+i)+a<<(k+j)=(a<<i)<<k+(a<<j)<<k=(a<<i+a<<j)<<k (3)
[0086] Among them, k is the common shift number, i and j are the independent shift numbers.
[0087] It is understandable that the system can be optimized hierarchically, and the shift operation can be divided into two steps. If the data to be processed needs to be shifted by 0 to 7 bits, it can be divided into two parts, namely, a primary shift and a secondary shift. For example, when performing a primary shift operation, 0 to 4 bits can be shifted, and when performing a secondary shift operation, 0 to 3 bits can be shifted. After the two-step shift operation, a shift of 0 to 7 bits can still be achieved.
[0088] Assuming that the target shift numbers are 4 and 6, the way to split the target shift numbers to obtain the common shift numbers and the independent shift numbers is as follows:
[0089] Because the shift number of the secondary shift in the hardware structure is 0 to 3, the extracted common coefficient should not be greater than 3. For example, when the target shift numbers are 4 and 6 in this example, the common shift number that can be extracted is 3, and the two independent shift numbers are: 4-3=1, 6-3=3.
[0090] Furthermore, in the neural network model, the data to be processed and the target data can be negative values. Therefore, in the process of data processing, in addition to shifting the data to be processed, sign processing can be further performed according to the signs of the data to be processed and the target weight to determine whether the sign bit of the processing result is positive or negative.
[0091] The data to be processed and the target weight both contain a sign bit. When the value of the sign bit is "0", it indicates that the sign is "+", and when the value of the sign bit is "1", it indicates that the sign is "-". The way to process the sign is to perform an inversion operation on the numerical sequence composed of the sign bit value of the data to be processed and the numerical values on each bit in the data to be processed, and then add 1, and perform a logical AND operation on the result and the numerical value of the sign bit of the target weight to obtain the symbol result of the symbol processing, and add the symbol result to the second shift result to obtain the processing result of the data to be processed.
[0092] Furthermore, before the data to be processed is shifted, the bit number of the data to be processed can be expanded, and according to the value of the sign bit of the data to be processed, a plurality of bit numbers of the sign bit value are added to the left side of the data to be processed. After the sign processing is performed, the obtained sign result can also be padded so that the bit number of the padded sign result is consistent with the bit number of the expanded data to be processed.
[0093] This application quantizes the weights into a finite number of powers of 2 (2 n ), thereby simplifying the multiplication operation into a finite number of shift and addition operations, thereby achieving the effect of optimizing the multiplication operation. At the same time, through hierarchical shift processing, the common factors (common shift numbers) of multiple parallel paths are extracted, and after the multi-path processing results are accumulated and merged, only one common factor operation is performed on the merged result. The multi-stage shifting method in the present application can reduce the number of bits for shift processing, so the hardware resources required for shift processing are relatively simple, and there is no need to perform multiple addition calculations on the data, thereby reducing the hardware resources of the adder.
[0094] In the present application examples, reference Figure 3 , the process of obtaining the target shift number based on the target weight is as follows:
[0095] S301: Based on the target weight, obtain a plurality of initial shift numbers.
[0096] Among them, each initial shift number is the value of all valid numerical sequences in the target weight.
[0097] For example, the initial weight after quantization is a=2 2 +2 3 +2 5 , then the valid numerical sequences in the target weight are the binary numbers corresponding to the value 2, the binary numbers corresponding to the value 3, and the binary numbers corresponding to the value 5, that is, the target weight is a=0010011101, and the initial shift numbers are 2, 3, and 5; the first 0 is the sign bit, and the subsequent 010 corresponds to the initial shift number 2, 011 corresponds to the initial shift number 3, and 101 corresponds to the initial shift number 5.
[0098] S302: Determine whether the difference between the initial shift amounts is less than a first preset difference.
[0099] If the differences between the initial shift numbers are all smaller than the first preset difference, execute S303; if the differences between the initial shift numbers are not smaller than the first preset difference, execute S304.
[0100] It should be noted that the first preset difference is not greater than the maximum shift number of the first-stage shift. Multiple executable first-stage shifts are provided in the hardware. If multiple first-stage shifts need to be shifted at the same time, it must be ensured that the difference between the shift numbers of each path is less than the first preset difference. The smaller the difference between the shift numbers, the process of the first-stage shifts of each path can be completed almost synchronously.
[0101] S303: Determine each initial shift number as the target shift number.
[0102] It can be understood that if the difference between the initial shift amounts is less than the first preset difference, it indicates that the initial shift amounts can be shifted almost synchronously during the shifting process.
[0103] S304: Split each initial shift number into multiple groups of target shift numbers.
[0104] The difference between each target shift number in each group of target shift numbers is smaller than the first preset difference.
[0105] It should be noted that after each initial shift number is split into multiple groups of target shift numbers, multiple rounds of the first strategy corresponding to steps S201 to S203 are executed for each target shift number in each group of target shift numbers. If multiple rounds of the first strategy need to be executed, the second shift results obtained by each execution of the first strategy are added together to obtain the processing result of the data to be processed.
[0106] For example, the initial shift numbers corresponding to the target weights are 1, 2, 6, and 7, and the first preset value is 4. At this time, there is a situation where the difference between the initial shift numbers is greater than the first preset value. Therefore, each initial shift number is divided into two groups of target shift numbers, the first group is 1, 2, and the second group is 6, 7; two rounds of the first strategy are executed on the data to be processed according to the grouping: the first strategy of the first round can be to split the target shift numbers 1 and 2 into a common shift number 1 and an independent shift number 0, 1, and perform a primary shift on the data to be processed according to the independent shift numbers 0 and 1, and then add them to obtain a first processing result, and then perform a secondary shift on the first processing result according to the common shift number 1 to obtain a second processing result; the first strategy of the second round can be to split the target shift numbers 6 and 7 into a common shift number 6 and an independent shift number 0, 1, and perform a primary shift on the data to be processed according to the independent shift numbers 0 and 1, and then add them to obtain a first processing result, and then perform a secondary shift on the first processing result according to the common shift number 6 to obtain a second processing result; finally, the second processing results obtained by the two rounds of execution of the first strategy are added to obtain the processing result of the data to be processed.
[0107] Optionally, before splitting each initial shift number, the number of target shift numbers for executing the first strategy in each round can be pre-set. For example, the target shift numbers are m, n, and k in ascending order; the counter number can be used to indicate how many components the first strategy in the first round can process. If number = 1, it means that only one can be processed in the first round, that is, m; then n and k are processed in the second round. If number = 2, it means that two can be processed in the first round, that is, m and n; then k is processed in the second round. If number = 3, it means that all three components have been processed in the first round and no second round of processing is required. In order to reduce hardware complexity, the value of number can be calculated in advance by software and stored in memory as a network parameter; or number can be calculated in real time by hardware when the system is running.
[0108] In the method provided in the embodiment of the present application, the data to be processed and the target weight are obtained, and the value of each group of valid numerical sequence in the target weight is obtained as the initial shift number according to the target weight. Determine whether the difference between each initial shift number is greater than the first preset value. If not, each initial shift number is the target shift number, and a round of the first strategy is executed, and the second shift result obtained in the first strategy is used as the processing result of the data to be processed; if so, each initial shift number is split into multiple groups, and the difference between the target shift numbers of each group is not greater than the first preset value, and multiple rounds of the first strategy are executed respectively according to the target shift numbers of each group split, and the second shift results obtained in each round of the first strategy are added to obtain the processing result of the data to be processed.
[0109] In the process of executing the first strategy, each target shift number is split into a common shift number and respective independent shift numbers according to the maximum shift number of a single shift operation supported by the hardware, the data to be processed is shifted once by multiple independent shift numbers and then added to obtain a first shift result, and the first shift result is shifted twice by the common shift number to obtain a second shift result.
[0110] The embodiment of the present application simplifies the multiplication process between the data to be processed and the weights in the neural network model by performing multiple shift operations. At the same time, during the shift operation, multiple shift numbers are split into multiple groups and then multi-stage shifts are performed, thereby reducing the complexity of the hardware shift operation. Performing multiple rounds of shifting can also reduce hardware resources.
[0111] The specific implementation processes and derivative methods of the above-mentioned embodiments are all within the protection scope of this application.
[0112] and Figure 1 Corresponding to the method described above, the embodiment of the present application further provides a data processing device for Figure 1 In order to implement the method in the embodiment of the present application, the data processing device provided in the embodiment of the present application can be applied to a computer terminal or various mobile devices, and its structural diagram is as follows: Figure 4 As shown, specifically including:
[0113] The first acquisition unit 401 is used to acquire the data to be processed and the target weight, wherein the target weight includes multiple groups of valid numerical sequences, and the values of each group of valid numerical sequences are the power values of each power of 2 after the initial weight is quantized into the sum of multiple powers of 2;
[0114] A second acquisition unit 402 is used to obtain at least one target shift number based on the target weight, each of which is a value of all valid numerical value sequences or a value of part of the valid numerical value sequences in the target weight;
[0115] The processing unit 403 is used to execute the first strategy and obtain a processing result of the data to be processed based on the obtained second shift result;
[0116] The first strategy includes:
[0117] Based on the maximum shift number of a single shift operation supported by the hardware, split each of the target shift numbers into the same common shift number and respective independent shift numbers;
[0118] The data to be processed are respectively shifted by one level by a plurality of independent shift numbers and then added to obtain a first shift result;
[0119] The first shift result is subjected to secondary shifting according to the common shift number to obtain a second shift result.
[0120] In the device provided in the embodiment of the present application, reference is made to Figure 5 ,according to Figure 5 The corresponding device can realize the process of shifting the data to be processed (IFM) according to each shift number corresponding to the weight (weight), and the processing unit 403 includes:
[0121] Multiple first-stage shift units 501, each first-stage shift unit 501 and each independent shift number (such as Figure 5 b0 and b1 in the figure correspond one to one, and are used to perform a first-order shift on the data to be processed according to the corresponding independent shift number;
[0122] An adding unit 502, used for adding the results after the first-stage shifting and outputting a first shifting result;
[0123] The secondary shift unit 503 is configured to perform secondary shift on the first shift result according to the common shift number, and output a second shift result.
[0124] The result unit 504 is used to output a processing result corresponding to the data to be processed according to the second shift result.
[0125] It should be noted that when executing the first strategy, the processing unit 403 can use the minimum target shift number as the common shift number, or use the common factor of each target shift number as the common shift number, wherein the common shift number is not greater than the maximum shift number supported by the secondary shift unit.
[0126] If the minimum target shift number is used as the common shift number, among the independent shift numbers, at least one independent shift number is 0. Figure 6 As shown, if the first-level shift unit 501 supports a shift of 0-4 bits (such as Figure 6 The secondary shift unit 503 supports a 0-3 bit shift (eg, Figure 6 0-3Shifter in the example, each target shift number is 1 and 4 respectively, then the value n=1 can be used as the common shift number, and each independent shift number is b0=0 and b1=3 respectively. At this time, only one first-level shift unit 501 can be applied in the first-level shift process. The first-level shift unit 501 shifts the data to be processed to the left by 3 bits. In the addition unit (Add) 502, the result of the first-level shift unit 501 is added to the unshifted data to be processed to obtain a first shift result. The second-level shift unit 503 performs a second-level shift on the first shift result. Finally, the result unit (Accumulator) 504 outputs the processing result corresponding to the data to be processed according to the second shift result of the second-level shift.
[0127] If the common factor is used as the common shift number, each independent shift number is greater than or equal to 0. Figure 7 As shown, if the first-level shift unit supports a shift of 0-4 bits, the second-level shift unit supports a shift of 0-3 bits, and the target shift numbers are 4 and 6 respectively, the numerical common factor 2 can be used as the common shift number, and the independent shift numbers are b0=2 and b1=4 respectively; at this time, two first-level shift units 501 are required to shift the data to be processed left by 2 bits and left by 4 bits respectively, the addition unit 502 adds the results of the two shifts and then the second-level shift unit 503 performs a second shift, and finally the result unit 504 outputs the processing result corresponding to the data to be processed according to the second shift result of the second shift.
[0128] Optional, reference Figure 6 and Figure 7 In the process of performing the primary shift and the secondary shift on the data to be processed, since the weights to be shifted and the data to be processed are signed data, the weights can be positive or negative, and the data to be processed can be positive or negative. Therefore, before the result unit processes the second shift result, it can obtain the sign of the weight and the sign of the data to be processed, and add the sign result obtained after calculating the sign to the second processing result to obtain the final processing result of the data to be processed.
[0129] Since the target shift number is split into two shifts, the number of bits that can be calculated in the set addition unit 502 can be relatively reduced. In the original shift process, taking int8 data as an example, if two int8 data need to be shifted, and the maximum shift number of each shift is 7 bits, then the required addition unit 502 needs to be able to process 16-bit data; applying the shift process of the present application, after splitting int8 into two shifts, after one-stage shift (the maximum shift number is a bit number smaller than 7 bits, for example: 4 bits), the shift number of the first-stage shift is less, and the number of bits that the addition unit 502 needs to process at this time is less (such as Figure 6 and Figure 7 The addition unit in the processor only needs to process 13 bits of data at most), which can further reduce hardware resources.
[0130] The specific working process of each unit in the data processing device disclosed in the above embodiment of the present application can be found in the corresponding content of the data processing method disclosed in the above embodiment of the present application, and will not be repeated here.
[0131] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative work.
[0132] Those skilled in the art may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, computer software, or a combination of both.
[0133] In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0134] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A data processing method, comprising: Obtaining data to be processed and a target weight, wherein the target weight includes multiple groups of valid numerical sequences, and the values of each group of valid numerical sequences are the power values of each power of 2 after the initial weight is quantized into the sum of multiple powers of 2; Based on the target weight, at least one target shift number is obtained, each of the target shift numbers being a value of all valid numerical value sequences or a value of a part of valid numerical value sequences in the target weight; Executing the first strategy, and obtaining a processing result of the data to be processed based on the obtained second shift result; The first strategy includes: Based on the maximum shift number of a single shift operation supported by the hardware, split each of the target shift numbers into the same common shift number and respective independent shift numbers; The data to be processed are respectively shifted by one level by a plurality of independent shift numbers and then added to obtain a first shift result; The first shift result is subjected to secondary shifting according to the common shift number to obtain a second shift result.
2. According to the method of claim 1, the first bit of the target weight is a sign bit, and the other bits are numerical bits, and each of the numerical bits includes multiple groups of valid numerical sequences.
3. The method according to claim 1, wherein obtaining a target shift number based on the target weight comprises: Based on the target weight, a plurality of initial shift numbers are obtained, each of which is a value of all valid value sequences in the target weight; Determining the difference between each of the initial shift numbers; If the differences between the initial shift numbers are all smaller than the first preset difference, the initial shift numbers are the target shift numbers; If the difference between the initial shift numbers is not less than the first preset difference, the initial shift numbers are divided into multiple groups of target shift numbers, and the difference between the target shift numbers in each group of target shift numbers is less than the first preset difference.
4. The method according to claim 3, wherein executing the first strategy comprises: If each of the initial shift numbers is divided into multiple groups of target shift numbers, multiple rounds of the first strategy are executed.
5. The method according to claim 4, wherein obtaining the processing result of the data to be processed based on the obtained second shift result comprises: The second shift results obtained by executing the first strategy each time are added together to obtain the processing result of the data to be processed. The method according to claim 1 , wherein the common shift number is less than or equal to the secondary shift number supported by the hardware. The method according to claim 1 , wherein the common shift number is a minimum shift number among the target shift numbers.
8. The method according to claim 1, wherein each of the independent shift numbers is not greater than a maximum shift number supported by the first-level shift, and the common shift number is not greater than a maximum shift number supported by the second-level shift.
9. A data processing device, comprising: A first acquisition unit is used to acquire the data to be processed and the target weight, wherein the target weight includes multiple groups of valid numerical sequences, and the values of each group of valid numerical sequences are the power values of each power of 2 after the initial weight is quantized into the sum of multiple powers of 2; A second acquisition unit is used to obtain at least one target shift number based on the target weight, each of which is a value of all valid numerical value sequences or a value of part of the valid numerical value sequences in the target weight; A processing unit, configured to execute the first strategy and obtain a processing result of the data to be processed based on the obtained second shift result; The first strategy includes: Based on the maximum shift number of a single shift operation supported by the hardware, split each of the target shift numbers into the same common shift number and respective independent shift numbers; The data to be processed are respectively shifted by one level by a plurality of independent shift numbers and then added to obtain a first shift result; The first shift result is subjected to secondary shifting according to the common shift number to obtain a second shift result.
10. The device according to claim 9, wherein the processing unit comprises: A plurality of primary shift units, each primary shift unit corresponding to each of the independent shift numbers one by one, and used to perform primary shift on the data to be processed according to the corresponding independent shift number; An adding unit, used for adding the results after the first-stage shift and outputting the first shift result; a secondary shift unit, configured to perform secondary shift on the first shift result according to the common shift number, and output a second shift result; A result unit is used to output a processing result corresponding to the data to be processed according to the second shift result.