Data processing method, accelerator and electronic device for neural network
By converting the objective function in the neural network into a Taylor polynomial and processing it in the accelerator, the problem of high computational complexity of activation function is solved, and the computing efficiency of the neural network is improved.
Patent Information
- Application Number
- CN202411198262.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-28
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-28
AI Technical Summary
The computational complexity of activation functions in neural networks leads to limited processing efficiency.
Convert the objective function to the target Taylor polynomial and use the pulsating array in the accelerator to perform multiplication and/or addition operation mode processing to improve operation efficiency.
Through the sum of multiple polynomials, the calculation of activation functions is simplified, and the computing efficiency of neural networks is significantly improved.
Smart Images

Figure CN118709729B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing of neural networks, and in particular to a data processing method, accelerator and electronic device for neural networks. Background Art
[0002] Neural network is a computational model that simulates the neural network of the human brain, and performs complex computational tasks through a large number of neurons and neural connections. In the operation process of neural network, activation functions are often used, such as Softmax, Sigmoid, ReLU, Tanh, etc., which are widely used in information transmission and nonlinear transformation between neurons, such as information transmission between neurons in neural network. Although activation functions play a very important role in the forward propagation process of neural network, their computational complexity is high and the processing efficiency of the data to be processed input into the neural network is limited. Summary of the invention
[0003] The purpose of the embodiments of the present application is to provide a data processing method, an accelerator and an electronic device for a neural network, which method can effectively improve the computing efficiency of the neural network.
[0004] In order to achieve the above object, the embodiment of the present application provides a data processing method for a neural network, comprising:
[0005] determining an objective function associated with operation of a first neural network, wherein the objective function is used to transfer information between neurons of the first neural network;
[0006] Based on a preset first association relationship, converting at least a portion of the target function into a corresponding target Taylor polynomial, wherein the target Taylor polynomial is operated based on a multiplication and / or addition operation mode;
[0007] Sending the target data to an accelerator for operating the first neural network, wherein the accelerator is provided with a systolic array, the systolic array includes at least one basic operation unit row, each of the basic operation unit rows includes a plurality of basic operation units connected in sequence, and the basic operation units respectively store weight data corresponding to each data item in the target Taylor polynomial;
[0008] The target data is processed using the basic operation unit row and based on the operation mode of the target Taylor polynomial and the weight data to determine a corresponding processing result.
[0009] Optionally, based on the preset first association relationship, converting at least part of the objective function into a corresponding target Taylor polynomial includes:
[0010] Based on the first association relationship, the expansion object in the objective function and the expansion order of the target Taylor polynomial are determined.
[0011] Optionally, a weight buffer is provided in the accelerator, and the weight buffer is respectively connected to the basic operation units arranged vertically in the systolic array, and the method further includes:
[0012] Based on the data structure of the target Taylor polynomial, determining weight data corresponding to each data item in the target Taylor polynomial;
[0013] The weight buffer is used to input each of the weight data into the result buffer of the vertically arranged basic operation unit corresponding thereto.
[0014] Optionally, the accelerator is provided with a feature buffer for outputting the target data, the feature buffer is respectively connected to each of the basic operation unit rows, the target data includes at least one feature sequence, each of the feature sequences includes a plurality of input feature values, each of the feature sequences corresponds to the basic operation unit row, and the target data is processed using the basic operation unit row and based on the operation mode of the target Taylor polynomial and the weight data to determine the corresponding processing result, including:
[0015] Inputting the feature sequence into the basic operation unit row corresponding thereto;
[0016] Perform product operations on each of the input feature values in the feature sequence and each of the weight data in the corresponding basic operation unit row step by step to generate corresponding product data;
[0017] The product data is stored in the result buffer and outputted using the result buffer.
[0018] Optionally, the accelerator is further provided with an output buffer, the output buffer is connected to the systolic array, the basic operation unit row is used, and based on the operation mode of the target Taylor polynomial and the weight data, the target data is processed to determine the corresponding processing result, and further includes:
[0019] Inputting the product data obtained by the row operations of each of the basic operation units into the output buffer;
[0020] The output buffer is used to perform an accumulation operation on the product data of the same basic operation unit row to determine a processing result of processing the corresponding feature sequence.
[0021] Optionally, the step of gradually performing a product operation on each of the input feature values in the feature sequence and each of the weight data in the corresponding basic operation unit row to generate corresponding product data includes:
[0022] Determining a correspondence between each of the input feature values and each of the basic operation units in the basic operation unit row;
[0023] Input the input feature value and the corresponding weight data in the result buffer of the basic operation unit into the multiplier-adder of the basic operation unit;
[0024] In the multiplier, the input feature value is multiplied by the corresponding weight data, and the product result is stored in the result buffer.
[0025] Optionally, the accelerator is further provided with a system bus and an array controller, the system bus is respectively connected to each module in the accelerator, and the array controller is used to control the systolic array.
[0026] The embodiment of the present application also provides a neural network accelerator, including:
[0027] A feature buffer, for determining an objective function associated with the operation of the first neural network, and based on a preset first association relationship, converting at least a portion of the objective function into a corresponding target Taylor polynomial, wherein the target Taylor polynomial is operated based on a multiplication and / or addition operation mode; the objective function is used to transfer information between neurons of the first neural network;
[0028] A weight buffer, used to determine weight data corresponding to each data item in the target Taylor polynomial;
[0029] The systolic array is connected to the feature buffer and the weight buffer respectively, and includes at least one basic operation unit row, each of which includes a plurality of basic operation units connected in sequence; the basic operation unit row is used to receive the weight data sent by the weight buffer, and based on the operation mode of the target Taylor polynomial and the weight data, process the target data to determine the corresponding processing result.
[0030] An embodiment of the present application further provides an electronic device, including: a processor and a memory, wherein an executable program is stored in the memory, and the memory executes the executable program to perform the steps of the method described above.
[0031] An embodiment of the present application also provides a computer program product, including a computer program / instruction, which implements the steps of the above method when the computer program / instruction is executed by a processor.
[0032] The processing method of the embodiment of the present application converts at least part of the objective function frequently used in the neural network into a corresponding target Taylor polynomial, and sets the pulsation array in the accelerator to a structure corresponding to the target Taylor polynomial, so that the pulsation array can be used to perform operations on the objective functions such as the activation function in the same operation mode as the target Taylor polynomial, thereby effectively improving the computational efficiency of the neural network. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 This is a flow chart of a data processing method for a neural network according to an embodiment of the present application;
[0034] Figure 2 A flowchart of a first embodiment of a data processing method according to an embodiment of the present application;
[0035] Figure 3 For the embodiments of this application Figure 1 A flowchart of an embodiment of step S400;
[0036] Figure 4 For the embodiments of this application Figure 3 A flowchart of an embodiment of step S420;
[0037] Figure 5 For the embodiments of this application Figure 1 A flowchart of another embodiment of step S400;
[0038] Figure 6 A schematic diagram of the structure of an accelerator according to an embodiment of the present application;
[0039] Figure 7 A schematic diagram of the structure of a systolic array according to an embodiment of the present application;
[0040] Figure 8 A schematic diagram of the structure of a basic computing unit in an embodiment of the present application;
[0041] Fig. 9 A schematic diagram of the steps of inputting a feature sequence of an embodiment of the present application into a basic operation unit for processing;
[0042] Fig.10 This is a structural block diagram of an accelerator according to an embodiment of the present application. DETAILED DESCRIPTION
[0043] Various aspects and features of the present application are described herein with reference to the accompanying drawings.
[0044] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of an embodiment. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.
[0045] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.
[0046] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.
[0047] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.
[0048] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.
[0049] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.
[0050] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.
[0051] A data processing method for a neural network in an embodiment of the present application can be applied to an accelerator of a neural network to improve the speed and efficiency of neural network calculations. The accelerator is used to process data related to a first neural network, including the operation of an objective function, which can be an activation function used or processed by the first neural network, such as Softmax, Sigmoid, ReLU, Tanh, etc. The activation function plays an important role in the forward propagation process of the first neural network.
[0052] like Figure 6As shown, the accelerator includes a system bus, a feature buffer, an array controller, an output buffer, a weight buffer and a systolic array, wherein the feature buffer, the array controller, the output buffer and the weight buffer are respectively connected to the system bus, and data can be transmitted through the system bus.
[0053] The systolic array is connected to the array controller, the output buffer and the weight buffer respectively. The feature buffer can send target data to the systolic array. The array controller is used to control the systolic array.
[0054] like Figure 7 As shown, the systolic array includes at least one basic operation unit row, each basic operation unit row includes a plurality of basic operation units (Control Unit, CU) connected in sequence, and the basic operation units respectively store weight data corresponding to each data item in the target Taylor polynomial converted by the target function. The target Taylor polynomial is a polynomial formed based on a multiplication and / or addition operation mode. The weight buffer is used for the weight data to be sent.
[0055] Based on the above structure of the accelerator, such as Figure 1 As shown, the data processing method of the embodiment of the present application includes the following steps:
[0056] S100, determining an objective function associated with an operation of a first neural network, wherein the objective function is used to transfer information between neurons of the first neural network.
[0057] Exemplarily, the first neural network is a computational model that simulates a human brain neural network, and performs complex computational tasks through a large number of neurons and neural connections. An objective function is used in the computation of the first neural network, and the objective function is used to transfer information between neurons of the first neural network. In one embodiment, the objective function is an activation function, and the activation function is used for information transfer and nonlinear transformation between neurons of the first neural network.
[0058] The objective function associated with the operation of the first neural network can be determined based on the working principle of the first neural network, operating data and / or empirical data. The objective function can be of one or more types.
[0059] S200, based on a preset first association relationship, converting at least a portion of the target function into a corresponding target Taylor polynomial, wherein the target Taylor polynomial is operated based on a multiplication and / or addition operation mode.
[0060] Exemplarily, when it is necessary to use an objective function to process target data, the target Taylor polynomial corresponding to the objective function can be used to implement operations on the target data, thereby improving operational efficiency.
[0061] Specifically, the target Taylor polynomial is operated based on a multiplication and / or addition operation mode. The target Taylor polynomial includes multiple data items, and multiplication and / or addition operations can be performed between the data items, so that the target Taylor polynomial can perform multiple similar simple operations, thereby improving the operation efficiency. For example, through the operation structure form of the target Taylor polynomial, the complex and unconventional activation equations in the neural network operation are converted into the operation form of the sum of multiple polynomials. In this way, the systolic array can improve the operation efficiency based on the operation form of the sum of multiple polynomials.
[0062] There is a first association relationship between the objective function and the target Taylor polynomial, and different objective functions can convert different target Taylor polynomials. When the objective function is needed to process the target data, the target Taylor polynomial corresponding to the objective function can be used to implement the operation on the target data.
[0063] For example, the objective function is the Softmax function, and the expression of the softmax function is:
[0064] (1)
[0065] Among them, the calculation process of the softmax function includes the exponential operation (exp) of the natural constant e, and the Taylor expansion is:
[0066] (2)
[0067] Therefore, the Softmax function has a preset first association relationship with the above formula (2). Based on the preset first association relationship, the Softmax function can be Convert to the corresponding operation form of the above formula (2).
[0068] Of course, other objective functions also have corresponding target Taylor polynomials. When converting the objective function, the relevant information of the pre-stored first association relationship can be called, and after determining the target Taylor polynomial corresponding to the objective function based on the first association relationship, at least part of the objective function is converted.
[0069] S300, sending the target data to an accelerator for operating the first neural network, wherein the accelerator is provided with a systolic array, the systolic array includes at least one basic operation unit row, each of the basic operation unit rows includes a plurality of basic operation units connected in sequence, and the basic operation units respectively store weight data corresponding to each data item in the target Taylor polynomial.
[0070] Exemplarily, the target data may be data that needs to be processed by the first neural network. After the target data is sent to an accelerator for operating the first neural network, the target data is sent to the systolic array of the accelerator through the feature buffer of the accelerator. The systolic array performs specific processing on the target data.
[0071] The systolic array includes at least one basic computing unit row, each basic computing unit row includes a plurality of basic computing units (CU) connected in sequence, and all basic computing units form an array. The systolic array has row attributes and column attributes. The row attributes include the number of basic computing units in each basic computing unit row, the connection relationship and other information, and the column attributes include the number of basic computing units in each column in the systolic array, the connection relationship and other information.
[0072] Specifically, multiple basic operation units in the basic operation unit row are connected in sequence, and when the target Taylor polynomial is processed using the basic operation unit row, multiple data units in the target data can be gradually input into the corresponding basic operation units in the basic operation unit row in sequence. Each basic operation unit stores weight data corresponding to each data item in the target Taylor polynomial.
[0073] In one embodiment, Figure 8 As shown, the basic operation unit includes a feature input unit, a multiplier, a comparator, a result cache and a result output unit connected in sequence; the basic operation unit also includes a weight input unit connected to the multiplier and a weight output unit connected to the weight input unit; the basic operation unit also includes a feature output unit connected to the feature input unit.
[0074] S400, using the basic operation unit row, and based on the operation mode of the target Taylor polynomial and the weight data, processing the target data to determine a corresponding processing result.
[0075] Exemplarily, the basic operation unit row includes a plurality of connected basic operation units, and the plurality of basic operation units correspond to respective data items in the target Taylor polynomial, so that the operation mode of the target Taylor polynomial can be implemented through the basic operation unit row.
[0076] More specifically, the operation mode of the target Taylor polynomial is to add multiple data items in the target Taylor polynomial, and the content of each data item may be different. In this embodiment, each value in the target data is sequentially input into the corresponding basic operation unit row, and an operation is performed based on the weight data stored in each basic operation unit, such as multiplying each value in the target data with the corresponding weight data and obtaining a corresponding result, thereby realizing the processing of the target data using the operation mode of the target Taylor polynomial.
[0077] In one embodiment, the systolic array includes a plurality of basic operation unit rows, each of which can process its own target data, and different basic operation unit rows can process different target data. For example, the target data includes: a feature sequence [1,3,3,3,3] and a feature sequence [1,1,1,1,1]. The feature sequence [1,3,3,3,3] can be sequentially input into the first basic operation unit row in the systolic array for operation, and the feature sequence [1,1,1,1,1] can be sequentially input into the second basic operation unit row in the systolic array for operation. The corresponding processing results are obtained respectively.
[0078] The processing method of the embodiment of the present application converts at least part of the objective function frequently used in the neural network into a corresponding target Taylor polynomial, and sets the pulsation array in the accelerator to a structure corresponding to the target Taylor polynomial, so that the pulsation array can be used to perform operations on the objective functions such as the activation function in the same operation mode as the target Taylor polynomial, thereby effectively improving the computational efficiency of the neural network.
[0079] In one embodiment of the present application, converting at least part of the objective function into a corresponding target Taylor polynomial based on the preset first association relationship comprises the following steps:
[0080] Based on the first association relationship, the expansion object in the objective function and the expansion order of the target Taylor polynomial are determined.
[0081] Exemplarily, there is a first association relationship between the objective function and the corresponding target Taylor polynomial, and the first association relationship represents the target Taylor polynomial corresponding to the objective function, and also represents the specific expansion object in the objective function, and the expansion object can be the entire objective function or a part of the objective function.
[0082] Each data item in the target Taylor polynomial may have its own expansion order. For example, in the above formula (2),
[0083]
[0084] The data items are , their respective expansion orders are 0, 1, 2, 3, etc.
[0085] Based on the first association relationship, each expansion order in the target Taylor polynomial after the target function is converted can be determined.
[0086] In one embodiment of the present application, the accelerator is provided with a weight buffer, and the weight buffer is respectively connected to the basic operation units arranged vertically in the systolic array, such as Figure 2 As shown, the method also includes:
[0087] S500, based on the data structure of the target Taylor polynomial, determine the weight data corresponding to each data item in the target Taylor polynomial.
[0088] Exemplarily, different target Taylor polynomials have their own data structures, which characterize the data representation form and data content of the target Taylor polynomial. Based on the data structure, the logical relationship between each data item in the target Taylor polynomial and the weight data of each data item are also specifically characterized. For example, the addition operation between the data items and the weight of each operation factor are characterized. Based on the data structure of the target Taylor polynomial, the weight data corresponding to each data item in the target Taylor polynomial can be determined.
[0089] Of course, different objective functions (such as activation functions) may have different weight data corresponding to each data item in the corresponding target Taylor polynomial.
[0090] S600, using the weight buffer, inputting each of the weight data into the result buffer of the basic operation unit arranged vertically corresponding to it.
[0091] Exemplarily, a weight buffer is provided in the accelerator, and weight data is stored in the weight buffer. The weight buffer is connected to the array controller and the systolic array respectively. Under the control of the array controller, the weight data is extracted from the weight buffer and input into the corresponding vertically arranged basic operation units, and specifically, the weight data is input into the result buffers of the basic operation units in the same column of the systolic array. This will enable the basic operation units in each column of the systolic array to have the same weight data.
[0092] In one embodiment of the present application, the accelerator is provided with a feature buffer for outputting the target data, the feature buffer is respectively connected to each of the basic operation unit rows, the target data includes at least one feature sequence, each of the feature sequences includes a plurality of input feature values, each of the feature sequences corresponds to the basic operation unit row, the target data is processed using the basic operation unit row and based on the operation mode of the target Taylor polynomial and the weight data to determine the corresponding processing result, such as Figure 3 As shown, including:
[0093] S410, inputting the feature sequence into the basic operation unit row corresponding thereto;
[0094] S420, performing a product operation on each of the input feature values in the feature sequence and each of the weight data in the corresponding basic operation unit row step by step to generate corresponding product data;
[0095] S430, storing the product data in the result buffer, and outputting using the result buffer.
[0096] Exemplarily, the feature buffer is used to send the target data to the systolic array of the accelerator. Specifically, the same target data can be sent to the same basic computing unit row. The target data includes at least one feature sequence, different feature sequences have different contents, and each feature sequence corresponds to a basic computing unit row.
[0097] For example, the target data includes a first feature sequence [1,3,3,3,3] and a second feature sequence [1,1,1,1,1]. The first feature sequence [1,3,3,3,3] can be sent to the corresponding first basic operation unit row, and the first basic operation unit row can process the feature sequence; the second feature sequence [1,1,1,1,1] can be sent to the corresponding second basic operation unit row, and the second basic operation unit row can process the feature sequence.
[0098] In one embodiment, each feature sequence includes multiple input feature values, for example, the input feature values in the first feature sequence [1, 3, 3, 3, 3] are 1, 3, 3, 3, 3, and the input feature values in the second feature sequence [1, 1, 1, 1, 1] are 1, 1, 1, 1, 1, 1.
[0099] After determining the input feature values in the feature sequence and the corresponding relationship between each input feature value and the basic operation unit, each input feature value is gradually multiplied with each weight data in the corresponding basic operation unit row to generate corresponding product data. Specifically, the weight data is stored in each basic operation unit in the basic operation unit row, and each input feature value can be multiplied with the weight data in the corresponding basic operation unit to generate corresponding product data. The product data is stored in the result buffer, and the result buffer is used to output the product data and then perform the accumulation operation.
[0100] Preferably, the input feature values in the feature sequence are multiplied step by step with the weight data in the corresponding basic operation unit row to generate corresponding product data, such as Figure 4 As shown, including:
[0101] S421, determining a correspondence between each of the input feature values and each of the basic operation units in the basic operation unit row;
[0102] S422, inputting the input feature value and the corresponding weight data in the result buffer of the basic operation unit into the multiplier-adder of the basic operation unit;
[0103] S423, in the multiplier, multiply the input feature value with the corresponding weight data, and store the product result in the result buffer.
[0104] Exemplarily, in the process of multiplying each input feature value in the feature sequence with the corresponding weight data, the corresponding relationship between each input feature value and the basic operation unit can be determined. In this embodiment, the same feature sequence is input into a single basic operation unit row, and after the input feature values in the feature sequence are arranged, they are sequentially input into each basic operation unit in the basic operation unit row, wherein there is a corresponding relationship between the corresponding input feature value and the basic operation unit.
[0105] Combination Figure 8 The result buffer of the basic operation unit stores weight data, and the weight data is input into the multiplier of the basic operation unit. In the multiplier, the input feature value corresponding to the basic operation unit is multiplied by the corresponding weight data, and the product result is stored in the result buffer of the basic operation unit.
[0106] The following is a description of a specific embodiment. Fig. 9 shown.
[0107] Step 1: Expand the target Taylor polynomial and determine that it is expanded to the fourth order. Accordingly, each basic operation unit row requires 5 CUs to process it.
[0108] Step 2: Send the input eigenvalues (coefficients) to the result buffer of the operation unit of the systolic array through the weight buffer in order of the corresponding times from large to small (1, 3, 3, 3, 3). The basic operation units in the same column share the same weight value. Among them, the weight value of the first column is 1 / 24, the weight value of the second column is 1 / 6, the weight value of the third column is 1 / 2, the weight value of the fourth column is 1, and the weight value of the fifth column is 1.
[0109] Step 3: Determine the first feature sequence [1,3,3,3,3] and the second feature sequence [1,1,1,1,1], each sequence occupies one basic operation unit row (one row of CU);
[0110] Among them, since there is a 0th-order term after the target Taylor polynomial is expanded, a 1 is added before the input eigenvalue in the feature sequence.
[0111] Step 4: The first input feature value (1) of the feature sequence is input to the first basic operation unit (first CU) of the first basic operation unit row through the feature buffer;
[0112] Step 5: The input feature value and the weight value (weight data) pre-stored in the result buffer are multiplied in the multiplier, and the product data is stored in the result buffer;
[0113] Step 6: After the operation, the feature input value is transferred to the next basic operation unit (next CU) through the feature output unit for further operation.
[0114] Based on the actual processing process, steps 4 to 6 may be repeated and the powers may be accumulated.
[0115] Step 7: Input eigenvalues and multiplication and addition calculations are the same as steps 2 to 5.
[0116] Step 8: When the feature sequence is empty, the calculation ends and the output of the product data is triggered. Specifically, the product data on the result buffer in the CU of each basic operation unit row is accumulated in the result output unit of the basic operation unit to obtain the processing result.
[0117] In one embodiment of the present application, the accelerator is further provided with an output buffer, the output buffer is connected to the systolic array, the basic operation unit row is used, and the target data is processed based on the operation mode of the target Taylor polynomial and the weight data to determine the corresponding processing result, such as Figure 5Shown and combined Fig. 9 , also includes:
[0118] S440, inputting the product data obtained by the row operations of each of the basic operation units into the output buffer.
[0119] Exemplarily, the output buffer is connected to the systolic array, and specifically can be connected to each column of the basic operation unit in the systolic array, to receive the product data output by the basic operation unit and perform accumulation operation on the product data.
[0120] S450, using the output buffer to perform an accumulation operation on the product data of the same basic operation unit row to determine a processing result for the corresponding feature sequence.
[0121] Herein, in combination with the above embodiment, after the first characteristic sequence [1,3,3,3,3] and the second characteristic sequence [1,1,1,1,1] are respectively input into the first basic operation unit row and the second basic operation unit row, the first product results are: 27 / 8, 9 / 2, 9 / 2, 3, 1; and the second product results are: 1 / 24, 1 / 6, 1 / 2, 1, 1.
[0122] The accumulation operation using the output buffer product data includes: accumulating 27 / 8, 9 / 2, 9 / 2, 3, 1, and accumulating 1 / 24, 1 / 6, 1 / 2, 1, 1. The processing results corresponding to the first feature sequence are 27 / 8+9 / 2+9 / 2+3+1, and the processing results corresponding to the second feature sequence are 1 / 24+1 / 6+1 / 2+1+1. The above processing results can simulate the processing results of the feature sequence (target data) using the target Taylor polynomial corresponding to the target function (such as activation function). Thereby, the processing efficiency is effectively improved on the basis of accurately determining the processing results.
[0123] In one embodiment of the present application, the accelerator is further provided with a system bus and an array controller, the system bus is respectively connected to each module in the accelerator, and the array controller is used to control the systolic array.
[0124] Exemplarily, the feature buffer, array controller, output buffer and weight buffer in the accelerator are respectively connected to the system bus, and data can be transmitted through the system bus. The array controller is used to control the systolic array, including: under the control of the array controller, extracting weight data from the weight buffer and inputting it into the basic operation units arranged vertically in the corresponding systolic array.
[0125] The present application also provides a neural network accelerator, such as Fig.10 As shown, including:
[0126] A feature buffer is used to determine an objective function associated with the operation of the first neural network, and based on a preset first association relationship, convert at least part of the objective function into a corresponding target Taylor polynomial, wherein the target Taylor polynomial is operated based on a multiplication and / or addition operation mode; the objective function is used to transfer information between neurons of the first neural network.
[0127] Exemplarily, the first neural network is a computational model that simulates a human brain neural network, and performs complex computational tasks through a large number of neurons and neural connections. An objective function is used in the computation of the first neural network, and the objective function is used to transfer information between neurons of the first neural network. In one embodiment, the objective function is an activation function, and the activation function is used for information transfer and nonlinear transformation between neurons of the first neural network.
[0128] The feature buffer may determine an objective function associated with the operation of the first neural network based on the working principle of the first neural network, operating data and / or empirical data. The objective function may be of one or more types.
[0129] When the target function needs to be used to process the target data, the feature buffer can use the target Taylor polynomial corresponding to the target function to implement the operation on the target data, thereby improving the operation efficiency.
[0130] Specifically, the target Taylor polynomial is operated based on a multiplication and / or addition operation mode. The target Taylor polynomial includes multiple data items, and multiplication and / or addition operations can be performed between the data items, so that the target Taylor polynomial can perform multiple similar simple operations, thereby improving the operation efficiency. For example, through the operation structure form of the target Taylor polynomial, the complex and unconventional activation equations in the neural network operation are converted into the operation form of the sum of multiple polynomials. In this way, the systolic array can improve the operation efficiency based on the operation form of the sum of multiple polynomials.
[0131] There is a first association relationship between the objective function and the target Taylor polynomial, and different objective functions can convert different target Taylor polynomials. When the objective function is needed to process the target data, the target Taylor polynomial corresponding to the objective function can be used to implement the operation on the target data.
[0132] A weight buffer is used to determine the weight data corresponding to each data item in the target Taylor polynomial.
[0133] The systolic array is connected to the feature buffer and the weight buffer respectively, and includes at least one basic operation unit row, each of which includes a plurality of basic operation units connected in sequence; the basic operation unit row is used to receive the weight data sent by the weight buffer, and based on the operation mode of the target Taylor polynomial and the weight data, process the target data to determine the corresponding processing result.
[0134] Exemplarily, the target data may be data that needs to be processed by the first neural network. After the target data is sent to an accelerator for operating the first neural network, the target data is sent to the systolic array of the accelerator through the feature buffer of the accelerator. The systolic array performs specific processing on the target data.
[0135] The systolic array includes at least one basic computing unit row, each basic computing unit row includes a plurality of basic computing units (CU) connected in sequence, and all basic computing units form an array. The systolic array has row attributes and column attributes. The row attributes include the number of basic computing units in each basic computing unit row, the connection relationship and other information, and the column attributes include the number of basic computing units in each column in the systolic array, the connection relationship and other information.
[0136] Specifically, multiple basic operation units in the basic operation unit row are connected in sequence. When the target Taylor polynomial is processed using the basic operation unit row, multiple data units in the target data can be gradually input into the corresponding basic operation units in the basic operation unit row in sequence. Each basic operation unit stores weight data corresponding to each data item in the target Taylor polynomial. The weight buffer is used to determine the weight data corresponding to each data item in the target Taylor polynomial.
[0137] In one embodiment, the basic operation unit includes a feature input unit, a multiplier, a comparator, a result cache and a result output unit connected in sequence; the basic operation unit also includes a weight input unit connected to the multiplier and a weight output unit connected to the weight input unit; the basic operation unit also includes a feature output unit connected to the feature input unit.
[0138] In one embodiment of the present application, the feature buffer is further used to:
[0139] Based on the first association relationship, the expansion object in the objective function and the expansion order of the target Taylor polynomial are determined.
[0140] In one embodiment of the present application, the weight buffers are respectively connected to the basic operation units arranged vertically in the systolic array, and the weight buffers are further used for:
[0141] Based on the data structure of the target Taylor polynomial, determining weight data corresponding to each data item in the target Taylor polynomial;
[0142] The weight buffer is used to input each of the weight data into the result buffer of the vertically arranged basic operation unit corresponding thereto.
[0143] In one embodiment of the present application, the accelerator is provided with a feature buffer for outputting the target data, the feature buffer is respectively connected to each of the basic operation unit rows, the target data includes at least one feature sequence, each of the feature sequences includes a plurality of input feature values, each of the feature sequences corresponds to the basic operation unit row, and the systolic array is further used for:
[0144] Inputting the feature sequence into the basic operation unit row corresponding thereto;
[0145] Perform product operations on each of the input feature values in the feature sequence and each of the weight data in the corresponding basic operation unit row step by step to generate corresponding product data;
[0146] The product data is stored in the result buffer and outputted using the result buffer.
[0147] In one embodiment of the present application, the accelerator is further provided with an output buffer, the output buffer is connected to the systolic array, and the systolic array is further used for:
[0148] Inputting the product data obtained by the row operations of each of the basic operation units into the output buffer;
[0149] The output buffer is used to perform an accumulation operation on the product data of the same basic operation unit row to determine a processing result of processing the corresponding feature sequence.
[0150] In one embodiment of the present application, the systolic array is further used to:
[0151] Determining a correspondence between each of the input feature values and each of the basic operation units in the basic operation unit row;
[0152] Input the input feature value and the corresponding weight data in the result buffer of the basic operation unit into the multiplier-adder of the basic operation unit;
[0153] In the multiplier, the input feature value is multiplied by the corresponding weight data, and the product result is stored in the result buffer.
[0154] In one embodiment of the present application, the accelerator is further provided with a system bus and an array controller, the system bus is respectively connected to each module in the accelerator, and the array controller is used to control the systolic array.
[0155] An embodiment of the present application further provides an electronic device, as shown in the figure, comprising a processor and a memory, wherein the memory stores an executable program, and the memory executes the executable program to perform the steps of the method described above.
[0156] An embodiment of the present application further provides a storage medium, wherein the storage medium carries one or more computer programs, and when the one or more computer programs are executed by a processor, the steps of the method described above are implemented.
[0157] An embodiment of the present application also provides a computer program product, including a computer program / instruction, which implements the steps of the above method when the computer program / instruction is executed by a processor.
[0158] It should be understood that in the embodiments of the present application, the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc.
[0159] It should also be understood that the memory mentioned in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM).
[0160] It should be noted that when the processor is a general-purpose processor, DSP, ASIC, FPGA or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, the memory (storage module) is integrated in the processor.
[0161] It should be noted that the memory described herein is intended to include, but is not limited to, these and any other suitable types of memory.
[0162] It should also be understood that the first, second, third, fourth and various numerical numbers involved in this document are only distinctions made for convenience of description and are not intended to limit the scope of the present application.
[0163] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.
[0164] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.
[0165] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.
[0166] Those skilled in the art will appreciate that the various illustrative logical blocks (ILBs) and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0167] In the several embodiments provided in the present application, it should be understood that the disclosed methods, accelerators and electronic devices can be implemented in other ways. For example, the accelerator embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0168] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0169] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0170] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.
[0171] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A data processing method for neural networks, It is characterized in that include: determining an objective function associated with operation of a first neural network, wherein the objective function is used to transfer information between neurons of the first neural network; Based on a preset first association relationship, converting at least a portion of the target function into a corresponding target Taylor polynomial, wherein the target Taylor polynomial is operated based on a multiplication and / or addition operation mode; Sending the target data to an accelerator for operating the first neural network, wherein the accelerator is provided with a systolic array, the systolic array includes at least one basic operation unit row, each of the basic operation unit rows includes a plurality of basic operation units connected in sequence, and the basic operation units respectively store weight data corresponding to each data item in the target Taylor polynomial; The target data is processed using the basic operation unit row and based on the operation mode of the target Taylor polynomial and the weight data to determine the corresponding processing result; wherein, The accelerator is provided with a weight buffer, and the weight buffer is respectively connected to the basic operation units arranged vertically in the systolic array, and the method further includes: Based on the data structure of the target Taylor polynomial, determining weight data corresponding to each data item in the target Taylor polynomial; Using the weight buffer, input each of the weight data into the result buffer of the basic operation unit arranged vertically corresponding to it, and the basic operation unit of each column in the systolic array has the same weight data; The target data includes at least one feature sequence, each of the feature sequences includes a plurality of input feature values, each of the feature sequences corresponds to the basic operation unit row, and the target data is processed using the basic operation unit row and based on the operation mode of the target Taylor polynomial and the weight data to determine a corresponding processing result, including: Inputting the feature sequence into the basic operation unit row corresponding thereto; Perform product operations on each of the input feature values in the feature sequence and each of the weight data in the corresponding basic operation unit row step by step to generate corresponding product data; The product data is stored in the result buffer and outputted using the result buffer.
2. The data processing method for a neural network according to claim 1, It is characterized in that The converting at least part of the objective function into a corresponding target Taylor polynomial based on the preset first association relationship comprises: Based on the first association relationship, the expansion object in the objective function and the expansion order of the target Taylor polynomial are determined.
3. The data processing method for a neural network according to claim 1, It is characterized in that The accelerator is provided with a feature buffer for outputting the target data, and the feature buffer is connected to each of the basic operation units respectively.
4. The data processing method for a neural network according to claim 3, It is characterized in that The accelerator is also provided with an output buffer, the output buffer is connected to the systolic array, the basic operation unit row is used, and based on the operation mode of the target Taylor polynomial and the weight data, the target data is processed to determine the corresponding processing result, and further includes: Inputting the product data obtained by the row operations of each of the basic operation units into the output buffer; The output buffer is used to perform an accumulation operation on the product data of the same basic operation unit row to determine a processing result of processing the corresponding feature sequence.
5. The data processing method for a neural network according to claim 3, It is characterized in that The step of gradually performing a product operation on each of the input feature values in the feature sequence and each of the weight data in the corresponding basic operation unit row to generate corresponding product data includes: Determining a correspondence between each of the input feature values and each of the basic operation units in the basic operation unit row; Input the input feature value and the corresponding weight data in the result buffer of the basic operation unit into the multiplier-adder of the basic operation unit; In the multiplier, the input feature value is multiplied by the corresponding weight data, and the product result is stored in the result buffer.
6. The data processing method for a neural network according to claim 1, It is characterized in that The accelerator is also provided with a system bus and an array controller. The system bus is connected to each module in the accelerator respectively, and the array controller is used to control the systolic array.
7. A neural network accelerator, It is characterized in that include: A feature buffer, for determining an objective function associated with the operation of the first neural network, and based on a preset first association relationship, converting at least a portion of the objective function into a corresponding target Taylor polynomial, wherein the target Taylor polynomial is operated based on a multiplication and / or addition operation mode; the objective function is used to transfer information between neurons of the first neural network; A weight buffer, used to determine weight data corresponding to each data item in the target Taylor polynomial; The systolic array is connected to the feature buffer and the weight buffer respectively, and includes at least one basic operation unit row, each of which includes a plurality of basic operation units connected in sequence; the basic operation unit row is used to receive the weight data sent by the weight buffer, and based on the operation mode of the target Taylor polynomial and the weight data, process the target data to determine the corresponding processing result; wherein, The weight buffers are respectively connected to the basic operation units arranged vertically in the systolic array, and the weight buffers are further used for: Based on the data structure of the target Taylor polynomial, determining weight data corresponding to each data item in the target Taylor polynomial; Using the weight buffer, input each of the weight data into the result buffer of the basic operation unit arranged vertically corresponding to it, and the basic operation unit of each column in the systolic array has the same weight data; The target data includes at least one feature sequence, each of the feature sequences includes a plurality of input feature values, each of the feature sequences corresponds to a row of basic operation units, and the systolic array is further used for: Inputting the feature sequence into the basic operation unit row corresponding thereto; Perform product operations on each of the input feature values in the feature sequence and each of the weight data in the corresponding basic operation unit row step by step to generate corresponding product data; The product data is stored in the result buffer and outputted using the result buffer.
8. An electronic device, It is characterized in that include: A processor and a memory, wherein an executable program is stored in the memory, and the memory executes the executable program to perform the steps of the method according to claim 1.
9. A computer program product, It is characterized in that The method comprises a computer program / instruction, which implements the steps of the method as claimed in claim 1 when the computer program / instruction is executed by a processor.
