Activation module for data calculation, AI processor and data calculation method

Abstract

The invention provides a data calculation activation module, an AI processor and a data calculation method, and the method comprises the steps: storing output calculation data to a cache unit when determining that a result calculated by a preorder matrix acceleration module meets an output condition; the read-write control unit reads the output calculation data from the cache unit and distributes the output calculation data to each calculation unit, softmax calculation is executed through each calculation unit, and a target calculation result is output. Thus, output calculation data output by the preorder matrix acceleration module can be stored in the cache unit of the activation module, softmax calculation is executed in parallel through at least one calculation unit after processing and data distribution of the read-write control unit, calculation, access and calling of all data are executed in the activation module, and the operation efficiency is improved. According to the method, the memory access overhead is reduced, the overall calculation performance is improved, and meanwhile the calculation efficiency can be improved through a parallel calculation processing scheme.

Description

Technical Field

[0001] This application relates to the field of electronic circuit technology, and in particular to an activation module for data computation, an AI processor, and a data computation method. Background Technology

[0002] With the rapid development of deep learning technology, the computational efficiency of other components in the network structure (such as convolutional layers and attention mechanisms) has been significantly improved. However, this overall performance evolution has also made the computational characteristics of the Softmax layer itself, especially when dealing with ultra-large-scale classification tasks (such as large language models), a bottleneck affecting the overall throughput and response speed of the system.

[0003] In existing technologies, deep neural network calculations can be performed using AI processors. Softmax calculations are typically performed by vector acceleration modules integrated within the AI ​​processor. Softmax operations must be performed in a specific order, completing a series of computational steps sequentially. Due to the complexity of Softmax calculations, the activation module cannot directly perform the Softmax calculation after the matrix acceleration module outputs its results. At this point, the relevant data must be stored in memory, and the vector acceleration module will then perform subsequent calculations. During the Softmax calculation process, multiple memory accesses are required to read or store data. This data transfer between memory and different processing units significantly increases memory access overhead, thereby reducing overall computational performance and impacting computational efficiency. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide an activation module for data computation, an AI processor, and a data computation method. The output computation data of the preceding matrix acceleration module can be stored in the cache unit of the activation module. After processing and data allocation by the read / write control unit, softmax computation is performed in parallel by at least one computing unit. All data computation and access are performed within the activation module, which reduces the overhead of memory access and improves the overall computing performance. At the same time, the parallel computing processing scheme can also improve computing efficiency.

[0005] In a first aspect, embodiments of this application provide an activation module for data computation, the activation module comprising a cache unit, a read / write control unit, and at least one computation unit; The cache unit is used to receive and store the output calculation data output by the preorder matrix acceleration module; the output calculation data is data calculated according to a specific output order. The read / write control unit is used to read the output calculation data from the cache unit and allocate the output calculation data to the calculation unit according to the specific output order; The computing unit is used to read the output computing data according to a preset processing cycle, perform softmax calculation on the allocated output computing data in parallel, and output the target computing result.

[0006] In one possible implementation, the output calculation data is a matrix containing X rows and N columns; each unit is assigned 1 row and N data points of output calculation data.

[0007] In one possible implementation, each of the computation units includes an extremum subunit, a subtractor, an exponentiation calculation subunit, an accumulator, a reciprocal subunit, and a multiplication subunit; The maximum / minimum subunit is used to read one line of N data from the read / write control unit, determine the maximum value among the N data, and send the maximum value to the subtractor. The subtractor is used to read one line of N data from the read / write control unit, calculate the difference between each data and the maximum value, obtain N target difference data, and send the N target difference data to the exponent calculation subunit. The exponent calculation subunit is used to receive N target difference data, perform exponent calculation on each target difference data to obtain N exponent calculation results, and send the N exponent calculation results to the read-write control unit and the accumulator so that the read-write control unit can store the N exponent calculation results in the cache unit. The accumulator is used to accumulate N exponential operation results to obtain an exponential accumulation result, and send the exponential accumulation result to the reciprocal subunit; The reciprocal subunit is used to perform a reciprocal calculation on the exponent accumulation result to obtain the reciprocal calculation result, and send the reciprocal calculation result to the multiplication subunit; The multiplication subunit is used to sequentially read N exponent operation results from the cache unit through the read-write control unit, perform calculations based on the reciprocal calculation result for each exponent operation result, determine N calculation sub-results, and output the target calculation result based on the N calculation sub-results.

[0008] In one possible implementation, when the multiplication subunit sequentially reads N exponentiation results from the read / write control unit, performs calculations based on the reciprocal calculation result for each exponentiation result, determines N sub-calculation results, and outputs the target calculation result based on the N sub-calculation results, the multiplication subunit is configured to: The N sub-results of each calculation unit are arranged according to the specific output order to output the initial calculation result; wherein, the initial calculation result is a matrix data containing X rows and N columns; The initial calculation results are aggregated according to a preset processing cycle, and the target calculation result is output; the target calculation result is a matrix containing a preset number of X rows and N columns of matrix data.

[0009] In one possible implementation, the final computational data output by the matrix acceleration module is a matrix containing M rows and N columns; where M is greater than X; the preset number is determined based on M and X.

[0010] In one possible implementation, the computing unit is further configured to: The target calculation result is saved to memory.

[0011] In one possible implementation, the number of computing units is consistent with the number of rows of matrix data in the output computing data; the activation module contains X computing units.

[0012] Secondly, embodiments of this application also provide an AI processor, the AI ​​processor including the activation module, matrix acceleration module and vector acceleration module described in any of the first aspects; The matrix acceleration module is used to perform convolution and matrix multiplication operations on the data to be processed according to a specific output order, obtain output calculation data, and send the output calculation data to the activation module. The activation module is used to receive the output calculation data sent by the matrix acceleration module, perform calculations on the output calculation data, and output the target calculation result. The vector acceleration module is used to read the output calculation data from memory and perform calculations on the output calculation data to obtain auxiliary calculation results.

[0013] In one possible implementation, the matrix acceleration module is further configured to: When it is determined that the output calculation data contains a matrix data of X rows and N columns, the output calculation data is sent to the activation module and the vector acceleration module, and the processing steps for the output calculation data are repeated until the final data output by the matrix acceleration module contains matrix data of M rows and N columns, which are all sent to the activation module and the vector acceleration module.

[0014] Thirdly, embodiments of this application also provide a data calculation method, applied to the activation module described in any of the first aspects, the data calculation method comprising: The cache unit is controlled to receive and store the output calculation data output by the preorder matrix acceleration module; the output calculation data is data calculated according to a specific output order. The read / write control unit is controlled to read the output calculation data from the cache unit and allocate the output calculation data to the calculation unit according to the specific output order; The calculation unit is controlled to read the output calculation data according to a preset processing cycle, perform softmax calculation on the allocated output calculation data in parallel, and output the target calculation result.

[0015] Fourthly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the data calculation method as described in any of the third aspects.

[0016] Fifthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the data calculation method as described in any of the third aspects.

[0017] The data computation activation module, AI processor, and data computation method provided in this application embodiment, when determining that the result of the calculation by the preceding matrix acceleration module meets the output conditions, store the output computation data in a cache unit; the read / write control unit reads the output computation data from the cache unit and allocates the output computation data to each computation unit, and performs softmax calculation through each computation unit to output the target computation result. In this way, the output computation data output by the preceding matrix acceleration module can be stored in the cache unit of the activation module. After processing and data allocation by the read / write control unit, softmax calculation is performed in parallel by at least one computation unit. All data computation and access are performed within the activation module, reducing memory access overhead and improving overall computational performance. Furthermore, the parallel computation processing scheme also improves computational efficiency.

[0018] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1This is a schematic diagram of the Softmax calculation process in the prior art provided in the embodiments of this application; Figure 2 This is one of the structural schematic diagrams of the activation module provided in the embodiments of this application; Figure 3 This is a second schematic diagram of the structure of the activation module provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the AI ​​processor provided in the embodiments of this application; Figure 5 This is a schematic diagram of the data processing flow of the matrix acceleration module provided in the embodiments of this application; Figure 6 A flowchart illustrating the data calculation method provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0021] Icons: 200 - Activation Module; 210 - Cache Unit; 220 - Read / Write Control Unit; 230 - Calculation Unit; 231 - Maximum / Minimum Value Subunit; 232 - Subtractor; 233 - Exponent Calculation Subunit; 234 - Accumulator; 235 - Reciprocal Subunit; 236 - Multiplication Subunit; 400 - AI Processor; 410 - Matrix Acceleration Module; 420 - Vector Acceleration Module; 700 - Electronic Device; 710 - Processor; 720 - Memory; 730 - Bus. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0023] First, the applicable scenarios for this application will be introduced. This application can be applied to the field of electronic circuit technology.

[0024] With the rapid development of deep learning technology, the computational efficiency of other components in the network structure (such as convolutional layers and attention mechanisms) has been significantly improved. However, this overall performance evolution has also made the computational characteristics of the Softmax layer itself, especially when dealing with ultra-large-scale classification tasks (such as large language models), a bottleneck affecting the overall throughput and response speed of the system.

[0025] In existing technologies, deep neural network computations can be performed using AI processors. AI processors are hardware chips specifically designed and optimized for artificial intelligence computing tasks. When handling AI workloads such as deep learning and machine learning, they offer higher energy efficiency and computational performance compared to traditional central processing units (CPUs) and graphics processing units (GPUs). Softmax computation is typically performed by a vector acceleration module integrated within the AI ​​processor. Softmax operations must be performed in a specific order, completing a series of computational steps sequentially. Due to the complexity of Softmax computation, after the matrix acceleration module completes its output, the activation module cannot directly perform the Softmax computation. At this point, the relevant data must be saved to memory, and the vector acceleration module then performs subsequent calculations.

[0026] For example, please refer to Figure 1 , Figure 1 This is a schematic diagram of the Softmax calculation process in the prior art provided in the embodiments of this application, such as... Figure 1 As shown, the Softmax calculation process is as follows: 1) Initial memory read and maximum value calculation: Perform the first memory read operation to obtain n input data x1,…,x from memory. n 1) Calculate the maximum value among these n input data, denoted as max(x); 2) Second memory read and difference calculation: Perform a second memory read to obtain these n input data again, and then calculate the difference between each input data and the maximum value max(x), i.e., xi max(x); 3) Exponentiation operation and result storage: For each difference x i Max(x) is used for exponentiation to obtain e. xi max(x) Then, the third memory access operation is performed to temporarily store the results of the exponentiation operation in memory. These results will be used as the numerators of subsequent division operations; 4) Accumulation operation: for the results e of n exponentiation operations... xi max(x) Perform an accumulation operation to calculate their sum, denoted as ∑(exi max(x) ), this result will be used as the denominator of the division operation; 5) Final division operation and result storage: Perform the 4th memory access, read the numerator (i.e. the result of the exponent operation) stored in step 3, and divide it with the sum obtained in step 4 to obtain the final output of the Softmax calculation. Finally, perform the 5th memory access to save the final output result in memory. This flow of data between memory and different processing units significantly increases memory access overhead, thereby reducing the overall computing performance and affecting computing efficiency.

[0027] In view of this, in the embodiments of this application, an activation module for data computation is provided, which can store the output computation data of the preceding matrix acceleration module in the cache unit of the activation module. After processing and data allocation by the read and write control unit, softmax computation is performed in parallel by at least one computing unit. All data computation and access are performed inside the activation module, which reduces the overhead of accessing memory and improves the overall computing performance. At the same time, the parallel computing processing scheme can also improve computing efficiency.

[0028] Furthermore, an activation module 200 for data calculation disclosed in an embodiment of this application will be introduced.

[0029] Please see Figure 2 , Figure 2 This is one of the structural schematic diagrams of the activation module 200 provided in the embodiments of this application, such as... Figure 2 As shown, the activation module 200 includes a cache unit 210, a read / write control unit 220, and at least one calculation unit 230. When it is determined that the result calculated by the preceding matrix acceleration module (not shown in the figure) meets the output conditions, the output calculation data is stored in the cache unit 210. The read / write control unit 220 reads the output calculation data from the cache unit and allocates the output calculation data to each calculation unit 230. The softmax calculation is performed by each calculation unit 230, and the target calculation result is output.

[0030] Specifically, the cache unit 210 is used to receive and store the output calculation data output by the preceding matrix acceleration module; the output calculation data is data calculated according to a specific output order.

[0031] In one possible implementation, the front-end matrix acceleration module outputs computational data in a specific order and stores it in the cache unit 210. Here, the output computational data stored in the cache unit 210 varies depending on the output order processed by the matrix acceleration module.

[0032] For example, taking the smallest granularity of the matrix acceleration module output as 16 data points, the output calculation data stored in the cache unit 210 can be 16 rows and N columns of data.

[0033] Specifically, the read / write control unit 220 is used to read the output calculation data from the cache unit 210 and allocate the output calculation data to the calculation unit 230 according to the specific output order.

[0034] In one possible implementation, the read / write control unit 220 reads and outputs the transposed data from the cache unit 210, and can also temporarily store the intermediate calculation results in the calculation unit 230, so that the calculation unit can quickly read the data during the calculation process and improve the data calculation efficiency.

[0035] Specifically, the calculation unit 230 is used to read the output calculation data according to a preset processing cycle, perform softmax calculation on the allocated output calculation data in parallel, and output the target calculation result.

[0036] In one possible implementation, the output computation data is a matrix containing X rows and N columns; each unit is allocated 1 row and N data points of output computation data; the number of computation units is consistent with the number of rows in the matrix data of the output computation data; the activation module contains X computation units, and multiple parallel computation units can simultaneously perform softmax calculations on multiple rows of data, thereby improving computational efficiency.

[0037] For example, taking the example that the smallest granularity of the matrix acceleration module output is 16 data points, the output calculation data stored in the cache unit 210 can be 16 rows and N columns of data, that is, the value of X is 16, and at this time the activation module 200 contains 16 calculation units.

[0038] In one possible implementation, the minimum granularity of output may vary for different matrix acceleration modules. The activation module provided in this application has good scalability and can be adapted to matrix acceleration modules with different minimum granularity of output, setting different numbers of computing units, for example, 4, 8 or 32, to flexibly meet different computing power requirements.

[0039] Here, Softmax normalization is a normalization method widely used in deep learning. Its core function is to transform a set of real numbers into a probability distribution, so that the sum of all output values ​​is 1 and each value is located in the (0,1) interval. It plays a key role in mapping the complex feature calculation results inside the network into an intuitive probability distribution.

[0040] Specifically, the Softmax function can be characterized by the following formula: ; Where max(x) is the maximum value among the N input data; x i max(x) is the difference between each input data point and the maximum value max(x); xi max(x) For each difference x i The result of exponential operation on max(x); The result e of the operation on N exponents xi max(x) The result of the cumulative operation.

[0041] In one possible implementation, according to the Softmax calculation formula, the calculation unit 230 needs to perform maximum and minimum value calculation, difference calculation, exponent calculation, summation calculation and division operation when performing Softmax calculation. Therefore, the calculation unit 230 also includes multiple sub-units, each performing different calculation processes.

[0042] Specifically, please refer to Figure 3 , Figure 3 This is a second schematic diagram of the structure of the activation module 200 provided in the embodiments of this application, as shown below. Figure 3 As shown, the activation module 200 includes a cache unit 210, a read / write control unit 220, and at least one calculation unit 230; each calculation unit 230 includes an extremum subunit 231, a subtractor 232, an exponent calculation subunit 233, an accumulator 234, a reciprocal subunit 235, and a multiplication subunit 236.

[0043] Specifically, the maximum / minimum subunit 231 is used to read one line of N data from the read / write control unit 220, determine the maximum value among the N data, and send the maximum value to the subtractor 232.

[0044] In one possible implementation, the extremum subunit 231 is responsible for calculating the maximum value in a row of data. Each clock cycle, the extremum subunit 231 obtains one data point from the read / write control unit 220. For the current row of N data points, it iterates through the data over N consecutive cycles to obtain the maximum value. After calculation, this maximum value is sent to the next-level subtractor 232. In the softmax calculation process, max(x) is obtained.

[0045] Specifically, the subtractor 232 is used to read one line of N data from the read / write control unit 220, calculate the difference between each data and the maximum value, obtain N target difference data, and send the N target difference data to the exponent calculation subunit 233.

[0046] In one possible implementation, after the maximum / minimum subunit 231 completes the calculation of the maximum value, the subtractor 232 starts the operation. At this time, the read / write control unit 220 reads another line of N data from the cache unit 210 and sends them to the subtractor 232 one by one. Each data is subtracted from the maximum value obtained in the previous stage, and one data is processed per clock cycle. The result is then sent to the next stage exponentiation calculation subunit 233. x is obtained in the softmax calculation process. i max(x).

[0047] Specifically, the exponent calculation subunit 233 is used to receive N target difference data, perform exponentiation on each target difference data to obtain N exponentiation results, and send the N exponentiation results to the read / write control unit 220 and the accumulator 234, so that the read / write control unit 220 can store the N exponentiation results in the cache unit 210.

[0048] In one possible implementation, the exponent calculation subunit 233 can perform exponent calculations via a lookup table. Specifically, the exponent calculation subunit 233 includes a programmable look-up table (LUT). By flexibly configuring the table entry parameters, it can efficiently perform nonlinear operations such as the exponential function (exp) and the linear rectified function (ReLU), as well as basic mathematical operations. The output path of the exponent calculation subunit 233 supports dual modes: one is that the data is received by the lower-level accumulator 234 to construct a pipeline processing channel; the other is that the data is transferred to the cache unit 210 via the read / write control unit 220 for data reuse. In the softmax calculation process, e is obtained... xi max(x) The results of N exponent operations need to be both added to the next-level accumulator (234) and temporarily stored in the cache unit (210).

[0049] In another possible implementation, the exponent calculation subunit 233 can also perform exponent calculation based on a dedicated exponent calculation circuit constructed using methods such as Newton's iteration method. Similarly, after obtaining the exponent calculation result, the N exponent calculation results are sent to the next-level accumulator 234 for accumulation and also to the cache unit 210 for temporary storage.

[0050] Specifically, the accumulator 234 is used to accumulate N exponential operation results to obtain an exponential accumulation result, and send the exponential accumulation result to the reciprocal subunit.

[0051] In one possible implementation, accumulator 234 can perform an accumulation and summation operation on the N consecutively output exponential operation results from the upper stage; this unit processes one exponential operation result per clock cycle, and completes the accumulation of all N exponential operation results after N consecutive cycles, and outputs the result to the lower-level reciprocal subunit 235. In the softmax calculation process, ∑(e...) xi max (x) ).

[0052] Specifically, the reciprocal subunit 235 is used to perform a reciprocal calculation on the exponent accumulation result to obtain the reciprocal calculation result, and send the reciprocal calculation result to the multiplication subunit 236.

[0053] In one possible implementation, the reciprocal subunit 235 can perform a reciprocal operation on the exponential accumulation result, and the result is sent to the multiplication subunit 236. In the softmax calculation process, 1 / ∑(ei)i is obtained. xi max(x) ).

[0054] Specifically, the multiplication subunit 236 is used to sequentially read N exponent operation results from the cache unit 210 through the read / write control unit 220, perform calculations based on the reciprocal calculation results for each exponent operation result, determine N calculation sub-results, and output the target calculation result based on the N calculation sub-results.

[0055] In one possible implementation, the multiplication subunit 236 may include two input ports. The first input port receives the reciprocal calculation result from the reciprocal subunit 235, and the second input port reads from the cache unit 210 via the read / write control unit 220. This unit completes one multiplication operation per clock cycle, and the result serves as the output of the entire calculation unit. In the softmax calculation process, it is also the final calculation result.

[0056] Here, as mentioned earlier, the currently active module contains multiple computing units, each of which can output a softmax calculation result. Multiple computing units need to be concatenated until all data in the matrix acceleration module has been processed.

[0057] Specifically, when the multiplication subunit 236 sequentially reads N exponentiation results from the read / write control unit 220, performs calculations based on the reciprocal calculation result for each exponentiation result, determines N sub-calculation results, and outputs the target calculation result based on the N sub-calculation results, the multiplication subunit 236 is used for: The N sub-results of each calculation unit are arranged according to the specific output order to output the initial calculation result; wherein, the initial calculation result is a matrix data containing X rows and N columns; The initial calculation results are aggregated according to a preset processing cycle, and the target calculation result is output; the target calculation result is a matrix containing a preset number of X rows and N columns of matrix data.

[0058] In one possible implementation, a 1-row N-column softmax calculation result can be obtained for each computing unit. Then, the 1-row N-column softmax calculation results obtained by multiple computing units 230 are concatenated to obtain an initial calculation result containing X-row N-column matrix data.

[0059] Furthermore, the calculation process is repeated, and the initial calculation results of the preset number of matrix data containing X rows and N columns are concatenated to obtain the target calculation result.

[0060] Here, the target calculation result is a matrix containing a preset number of X rows and N columns of matrix data; the final calculation data output by the matrix acceleration module is a matrix containing M rows and N columns; where M is greater than X; the preset number is determined based on M and X.

[0061] For example, taking the smallest granularity of the matrix acceleration module output as 16 data points, the output calculation data stored in the cache unit 210 each time can be 16 rows and N columns of data. That is, the initial calculation result of each output is a matrix data of 16 rows and N columns. At this time, the final calculation data output by the matrix acceleration module is a matrix data of 128 rows and N columns. It is necessary to perform 8 calculation processes. The final target calculation result is a matrix data of 128 rows and N columns obtained by concatenating 8 matrix data of 16 rows and N columns.

[0062] In another possible implementation, the reciprocal subunit 235 and the multiplication subunit 236 in the activation module 200 are cascaded to perform the division operation. This design is efficient in hardware implementation, but it is essentially one path to implement the division function. This combined structure can also be directly replaced by a single dedicated division calculation subunit.

[0063] Furthermore, after the calculation unit 230 in the activation module 200 outputs the target calculation result, the obtained target calculation result can be saved to memory so that subsequent processing can continue.

[0064] Specifically, the computing unit is also used to: save the target calculation result to memory.

[0065] In this way, without accessing external main memory, the data is temporarily stored in the cache of the activation module, and all the steps required for softmax calculation are completed in parallel by the multiple sets of computing units. Only the final result is written back, which reduces the overhead of accessing memory and improves the overall computing performance.

[0066] Furthermore, in one possible implementation, the activation module 200 is located within the AI ​​processor and can collaborate with other modules to perform data processing. For details, please refer to [link to relevant documentation]. Figure 4 , Figure 4 This is a schematic diagram of the structure of the AI ​​processor 400 provided in the embodiments of this application, as shown below. Figure 4 As shown, the AI ​​processor 400 includes an activation module 200, a matrix acceleration module 410, and a vector acceleration module 420.

[0067] Specifically, the matrix acceleration module 410 has high-efficiency processing capabilities, can read data from memory, and quickly complete convolution and matrix multiplication operations; the activation module 200 is usually located downstream of the matrix acceleration module 410, and can directly receive the results of matrix multiplication operations to perform simple operations such as addition, ReLU, and Softmax calculation; the vector acceleration module 420 can perform auxiliary operations.

[0068] Specifically, the matrix acceleration module 410 is used to perform convolution and matrix multiplication operations on the data to be processed according to a specific output order to obtain output calculation data, and send the output calculation data to the activation module 200.

[0069] In one possible implementation, the matrix acceleration module 410 can process multiple output channels, multiple input channels, and multiple output points simultaneously in parallel. For example, for matrix multiplication, a strategy of parallel operation of multiple left matrix rows, multiple right matrix rows, and multiple result matrix columns can be adopted.

[0070] For example, please refer to Figure 5 , Figure 5 This is a schematic diagram of the data processing flow of the matrix acceleration module provided in the embodiments of this application, as shown below. Figure 5As shown, assume the final generated matrix has a size of m rows and n columns, and the parallelism of the matrix acceleration module in the column direction of the resulting matrix is ​​set to 16, denoted as n for ease of description. 16 That is, the smallest granularity of the output is 16 data points. In the process of outputting the result matrix, the matrix acceleration module follows specific output rules: first, according to... Figure 5 In the first order, each clock cycle outputs the data from column 1 to column 16 of a row sequentially. After 16 clock cycles, the data from column 1 to column 16 of the first 16 rows will have been output, denoted as m. 16 n 16 Following the second sequence, in the next clock cycle, the matrix acceleration module outputs the data from columns 17 to 32 of the first row. After several clock cycles, all data from the first 16 rows is output, denoted as n' and m'. 16 n 16 Where n` is the quotient of n and 16. Then, starting from the third order, output the data from columns 1 to 16 of row 17, repeating this cycle until all data in the entire result matrix has been output. The output order of the entire matrix is ​​denoted as m`, n`, m. 16 n 16 , where m` is the quotient of m and 16.

[0071] In one possible implementation, after the matrix acceleration module 410 calculates the entire row of data, it can send the obtained output calculation data to the activation module 200 and the vector acceleration module 420 to perform calculations synchronously.

[0072] Specifically, the matrix acceleration module is further configured to: when it is determined that the output calculation data contains matrix data with X rows and N columns, send the output calculation data to the activation module and the vector acceleration module, and repeat the processing steps for the output calculation data until all the matrix data with M rows and N columns contained in the final data output by the matrix acceleration module are sent to the activation module and the vector acceleration module.

[0073] Here, taking the smallest granularity of the matrix acceleration module 410 output as 16 data points as an example, that is, X can be 16. After determining that the output calculation data contains matrix data with 16 rows and N columns, the output calculation data is sent to the activation module 200 and the vector acceleration module 420. After determining that the output calculation data has been sent to the activation module 200 and the vector acceleration module 420 for processing is complete, the output calculation data containing matrix data with 16 rows and N columns is output again. The processing steps for the output calculation data are repeated until the final data output by the matrix acceleration module 410 contains matrix data with M rows and N columns, which are all sent to the activation module 200 and the vector acceleration module 420, and the data processing flow ends.

[0074] In one possible implementation, after the activation module 200 receives the output calculation data from the matrix acceleration module 410, it can perform various calculations, including softmax, on the output calculation data, and then output the target calculation result. The specific determination process of the target calculation result is consistent with the detailed processing process in the activation module 200 described above, and will not be repeated here.

[0075] In one possible implementation, after the vector acceleration module 420 reads the output calculation data from memory, it performs calculations other than softmax calculation. Alternatively, the vector acceleration module 420 can assist in the calculation of softmax calculation, which cannot be handled by the activation module 200, thereby further improving the data calculation efficiency.

[0076] The data computation activation module provided in this embodiment stores the output computation data in a cache unit when it determines that the result of the preceding matrix acceleration module's computation meets the output conditions. The read / write control unit reads the output computation data from the cache unit and allocates it to each computation unit. Each computation unit then performs a softmax calculation and outputs the target computation result. In this way, the output computation data from the preceding matrix acceleration module can be stored in the activation module's cache unit. After processing and data allocation by the read / write control unit, the softmax calculation is performed in parallel by at least one computation unit. All data computation and access are performed within the activation module, reducing memory access overhead and improving overall computational performance. Furthermore, the parallel computation processing scheme also improves computational efficiency.

[0077] Based on the same inventive concept, this application also provides a data calculation method for the activation module. Since the principle of the method in this application is similar to that of the activation module for data calculation in the above-mentioned embodiment of this application, the implementation of the method can be referred to the implementation of the method, and the repeated parts will not be described again.

[0078] Please see Figure 6 , Figure 6 A flowchart of the data calculation method provided in the embodiments of this application is shown below. Figure 6 As shown, the data calculation method includes: S601, Control the cache unit to receive and store the output calculation data output by the preorder matrix acceleration module; the output calculation data is data calculated according to a specific output order.

[0079] S602. Control the read / write control unit to read the output calculation data from the cache unit and allocate the output calculation data to the calculation unit according to the specific output order.

[0080] S603. Control the calculation unit to read the output calculation data according to a preset processing cycle, perform softmax calculation on the allocated output calculation data in parallel, and output the target calculation result.

[0081] In one possible implementation, the output calculation data is a matrix containing X rows and N columns; each unit is assigned 1 row and N data points of output calculation data.

[0082] In one possible implementation, each of the calculation units includes an extremum subunit, a subtractor, an exponentiation calculation subunit, an accumulator, a reciprocal subunit, and a multiplication subunit; the control unit reads the output calculation data according to a preset processing cycle, performs softmax calculation on the allocated output calculation data in parallel, and outputs the target calculation result, including: The maximum / minimum subunit is controlled to read one line of N data from the read / write control unit, determine the maximum value among the N data, and send the maximum value to the subtractor; The subtractor is controlled to read one line of N data from the read / write control unit, and calculates the difference between each data and the maximum value to obtain N target difference data, and sends the N target difference data to the exponent calculation subunit; The control unit receives N target difference data and performs exponential operation on each target difference data to obtain N exponential operation results. The N exponential operation results are then sent to the read / write control unit and the accumulator so that the read / write control unit can store the N exponential operation results in the cache unit. The accumulator is controlled to accumulate N exponential operation results to obtain an exponential accumulation result, and the exponential accumulation result is sent to the reciprocal subunit; The reciprocal subunit is controlled to perform a reciprocal calculation on the exponent accumulation result to obtain the reciprocal calculation result, and the reciprocal calculation result is sent to the multiplication subunit; The control unit reads N exponent operation results sequentially from the cache unit through the read / write control unit. For each exponent operation result, it performs calculations based on the reciprocal calculation result to determine N calculation sub-results. Based on the N calculation sub-results, it outputs the target calculation result.

[0083] In one possible implementation, the control unit reads N exponentiation results sequentially from the cache unit via the read / write control unit, performs calculations based on the reciprocal calculation result for each exponentiation result, determines N sub-results, and outputs the target calculation result based on the N sub-results, including: The N sub-results of each calculation unit are arranged according to the specific output order to output the initial calculation result; wherein, the initial calculation result is a matrix data containing X rows and N columns; The initial calculation results are aggregated according to a preset processing cycle, and the target calculation result is output; the target calculation result is a matrix containing a preset number of X rows and N columns of matrix data.

[0084] In one possible implementation, the final computational data output by the matrix acceleration module is a matrix containing M rows and N columns; where M is greater than X; the preset number is determined based on M and X.

[0085] In one possible implementation, the data calculation method further includes: The target calculation result is saved to memory.

[0086] In one possible implementation, the number of computing units is consistent with the number of rows of matrix data in the output computing data; the activation module contains X computing units.

[0087] The AI ​​processor provided in this application, when determining that the result calculated by the preceding matrix acceleration module meets the output conditions, stores the output computation data in a cache unit. The read / write control unit reads the output computation data from the cache unit and allocates the output computation data to each computation unit. Each computation unit then performs a softmax calculation and outputs the target computation result. In this way, the output computation data output by the preceding matrix acceleration module can be stored in the cache unit of the activation module. After processing and data allocation by the read / write control unit, the softmax calculation is performed in parallel by at least one computation unit. All data calculations and access are performed within the activation module, reducing memory access overhead and improving overall computational performance. Furthermore, the parallel computational processing scheme also improves computational efficiency.

[0088] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device 700 includes a processor 710, a memory 720, and a bus 730.

[0089] The memory 720 stores machine-readable instructions executable by the processor 710. When the electronic device 700 is running, the processor 710 communicates with the memory 720 via the bus 730. When the machine-readable instructions are executed by the processor 710, they can perform the operations described above. Figure 6 The steps of the data calculation method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0090] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 6 The steps of the data calculation method in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0091] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0093] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0095] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An activation module for data computation, characterized by, The activation module comprises a cache unit, a read-write control unit and at least one calculation unit; The cache unit is configured to receive and store output calculation data output by a preceding matrix acceleration module; the output calculation data is data calculated according to a specific output sequence; The read-write control unit is configured to read the output calculation data from the cache unit and distribute the output calculation data to the calculation unit according to the specific output sequence; The calculation unit is configured to read the output calculation data according to a preset processing period, perform softmax calculation on the distributed output calculation data in parallel, and output a target calculation result.

2. The activation module of claim 1, wherein, The output calculation data comprises X rows and N columns of matrix data; and the output calculation data distributed to each calculation unit comprises one row and N data.

3. The activation module of claim 2, wherein, Each calculation unit comprises a maximum value subunit, a subtracter, an exponential calculation subunit, an accumulator, an inverse subunit and a multiplication subunit; The maximum value subunit is configured to read one row and N data from the read-write control unit, determine a maximum value in the N data, and send the maximum value to the subtracter; The subtracter is configured to read one row and N data from the read-write control unit, calculate a difference between each data and the maximum value one by one, obtain N target difference data, and send the N target difference data to the exponential calculation subunit; The exponential calculation subunit is configured to receive the N target difference data, perform exponential operation on each target difference data, obtain N exponential operation results, and send the N exponential operation results to the read-write control unit and the accumulator, so that the read-write control unit stores the N exponential operation results in the cache unit; The accumulator is configured to accumulate the N exponential operation results to obtain an exponential accumulation result, and send the exponential accumulation result to the inverse subunit; The inverse subunit is configured to perform inverse calculation on the exponential accumulation result to obtain an inverse calculation result, and send the inverse calculation result to the multiplication subunit; The multiplication subunit is configured to sequentially read the N exponential operation results from the cache unit through the read-write control unit, perform calculation on each exponential operation result based on the inverse calculation result, determine N calculation subresults, and output the target calculation result based on the N calculation subresults.

4. The activation module of claim 3, wherein, When the multiplication subunit is configured to sequentially read the N exponential operation results from the read-write control unit, perform calculation on each exponential operation result based on the inverse calculation result, determine N calculation subresults, and output the target calculation result based on the N calculation subresults, the multiplication subunit is configured to: arrange the N calculation subresults of each calculation unit according to the specific output sequence to output an initial calculation result; wherein the initial calculation result comprises X rows and N columns of matrix data; perform set on the obtained initial calculation result according to a preset processing period, and output the target calculation result; the target calculation result comprises a preset number of X rows and N columns of matrix data.

5. The activation module of claim 4, wherein, The final calculation data output by the matrix acceleration module is matrix data containing M rows and N columns; wherein M is greater than X; the preset number is determined according to M and X.

6. The activation module of claim 1, wherein, The calculation unit is further configured to: save the target calculation result to the memory.

7. The activation module of claim 2, wherein, The number of the calculation units is consistent with the number of rows of the matrix data in the output calculation data; the activation module contains X calculation units.

8. An AI processor, comprising: The AI processor comprises the activation module, the matrix acceleration module and the vector acceleration module according to any one of claims 1-7. The matrix acceleration module is configured to perform convolution operation and matrix multiplication calculation on the to-be-processed data according to a specific output sequence to obtain output calculation data, and send the output calculation data to the activation module. The activation module is configured to receive the output calculation data sent by the matrix acceleration module, and perform calculation on the output calculation data to output target calculation result. The vector acceleration module is configured to read the output calculation data from the memory, and perform calculation on the output calculation data to obtain auxiliary calculation result.

9. The Al processor of claim 8, wherein, The matrix acceleration module is further configured to: when it is determined that the output calculation data contains matrix data of X rows and N columns, send the output calculation data to the activation module and the vector acceleration module, and loop the processing steps for the output calculation data until the final data output by the matrix acceleration module contains matrix data of M rows and N columns are all sent to the activation module and the vector acceleration module.

10. A data computing method, characterized by, The data calculation method is applied to the activation module according to any one of claims 1-7, and the data calculation method comprises: controlling the cache unit to receive and store the output calculation data output by the previous matrix acceleration module; the output calculation data is data calculated according to a specific output sequence; controlling the read-write control unit to read the output calculation data from the cache unit, and distributing the output calculation data to the calculation units according to the specific output sequence; controlling the calculation unit to read the output calculation data according to a preset processing period, and performing softmax calculation on the distributed output calculation data in parallel to output target calculation result.

Images