A matrix data processing method, device, electronic device and storage medium

Abstract

The present invention discloses a method for processing matrix data, including: obtaining matrix data to be processed and weight matrix data of a recursive neural network; matching element parallel parameters and vector parallel parameters corresponding to the scale parameters according to the scale parameters of the recursive neural network; dividing the weight matrix data into blocks according to the element parallel parameters and the vector parallel parameters to obtain weight vector blocks; multiplying the weight vector blocks with the matrix elements to be processed to obtain a first processing result; configuring an addition tree tail according to the element parallel parameters, accumulating the first processing result through an addition tree including the addition tree tail to obtain a second processing result, and outputting the second processing result as the processing result of the matrix data to be processed. The utilization rate of hardware resources is improved, the slicing strategy can be made more flexible, and it can be applied to recursive neural networks of various sizes.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and more specifically, to a matrix data processing method, device, electronic device and storage medium. Background Art

[0002] With the rapid development of machine learning, recurrent neural networks (RNNs) have been shown to have useful properties for many important applications. Because they can record previous information to improve prediction accuracy, RNNs are applied to speech recognition, natural language processing, and video classification, and many variants have been developed. Among the many RNN variants, the two most popular are long short-term memory networks (LSTM) and gated recurrent units (GRU). However, data dependencies in RNN calculations stall the system until the required hidden vector is returned from the full pipeline to start the next time step calculation, such as Figure 1a As shown, in Figure 1a In the example above, the hidden vector h t The next time step calculation starts only after returning. Moreover, a deeper pipeline is usually used to achieve a higher operating frequency, which makes the stagnation loss worse because the system pipeline needs to be cleared, and the hardware resources are idle during the stagnation process. Therefore, the existing RNN has a low utilization rate of hardware resources, and RNNs of a certain scale also have requirements for hardware resources and are not very flexible.

[0003] Application Contents

[0004] The purpose of the present invention is to provide a matrix data processing method to improve the utilization rate of RNN for hardware resources in view of the defects of the above-mentioned prior art, and to further improve the utilization rate of RNN for hardware resources by flexibly configuring hardware resources.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] In a first aspect, a method for processing matrix data for a recursive neural network is provided, the method comprising:

[0007] Obtaining matrix data to be processed and weight matrix data of the recursive neural network, wherein both the matrix data to be processed and the weight matrix data are composed of matrix elements, and the matrix data includes column vectors constructed by the matrix elements;

[0008] According to a scale parameter of the recursive neural network, matching an element parallel parameter and a vector parallel parameter corresponding to the scale parameter;

[0009] Divide the weight matrix data into blocks according to the element parallel parameter and the vector parallel parameter to obtain weight vector blocks;

[0010] Perform multiplication calculation on the weight vector block and the matrix element to be processed to obtain a first processing result;

[0011] The addition tree tail is configured according to the element parallel parameter, the first processing result is accumulated through the addition tree including the addition tree tail to obtain a second processing result, and the second processing result is output as the processing result of the matrix data to be processed.

[0012] Optionally, configuring the addition tree tail according to the element parallel parameter includes:

[0013] Arrange the addition tree tail behind the addition tree;

[0014] Configure parallelism of the addition tree tail.

[0015] Optionally, the scale parameter of the recursive neural network includes: the number of processing units and the vector dimension, and the matching of the element parallelism parameter and the vector parallelism parameter corresponding to the scale parameter according to the scale parameter of the recursive neural network includes:

[0016] According to the number of processing units and the vector dimension, element parallelism parameters and vector parallelism parameters corresponding to the number of processing units and the vector dimension are matched.

[0017] Optionally, before accumulating the first processing result through the addition tree including the addition tree tail to obtain the second processing result, the method further includes:

[0018] A balance calculation is performed on the first processing result to balance the parallelism of the weight vector blocks.

[0019] In a second aspect, an embodiment of the present invention further provides a matrix data processing device for a recursive neural network, the device comprising:

[0020] An acquisition module, used for acquiring the to-be-processed matrix data and weight matrix data of the recursive neural network, wherein the to-be-processed matrix data and the weight matrix data are both composed of matrix elements, and the matrix data includes column vectors constructed by the matrix elements;

[0021] A matching module, configured to match element parallel parameters and vector parallel parameters corresponding to the scale parameter of the recursive neural network according to the scale parameter of the recursive neural network;

[0022] A processing module, used for dividing the weight matrix data into blocks according to the element parallel parameters and the vector parallel parameters to obtain weight vector blocks;

[0023] A calculation module, used for performing multiplication calculation on the weight vector block and the matrix element to be processed to obtain a first processing result;

[0024] An output module is used to configure an addition tree tail according to an element parallel parameter, accumulate the first processing result through an addition tree including the addition tree tail to obtain a second processing result, and output the second processing result as a processing result of the matrix data to be processed.

[0025] Optionally, the output module includes:

[0026] A first configuration unit, configured to configure the addition tree tail behind the addition tree;

[0027] The second configuration unit is used to configure the parallelism of the addition tree tail.

[0028] Optionally, the scale parameters of the recursive neural network include: the number of processing units and the vector dimension, and the matching module is further used to match the element parallel parameters and vector parallel parameters corresponding to the number of processing units and the vector dimension according to the number of processing units and the vector dimension.

[0029] Optionally, the device further comprises:

[0030] A balancing module is used to perform a balance calculation on the first processing result to balance the parallelism of the weight vector blocks.

[0031] In a third aspect, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the matrix data processing method provided in an embodiment of the present invention when executing the computer program.

[0032] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the matrix data processing method provided in an embodiment of the present invention are implemented.

[0033] The beneficial effects brought by the present invention are as follows: obtaining the to-be-processed matrix data and weight matrix data of the recursive neural network, wherein the to-be-processed matrix data and weight matrix data are both composed of matrix elements, and the matrix data include column vectors constructed by the matrix elements; according to the scale parameter of the recursive neural network, matching the element parallel parameters and vector parallel parameters corresponding to the scale parameter; dividing the weight matrix data into blocks according to the element parallel parameters and the vector parallel parameters to obtain weight vector blocks; multiplying the weight vector blocks with the to-be-processed matrix elements to obtain a first processing result; configuring the addition tree tail according to the element parallel parameters, accumulating the first processing result through the addition tree containing the addition tree tail to obtain a second processing result, and outputting the second processing result as the processing result of the to-be-processed matrix data. By multiplying the column vector of the weight matrix data with the matrix elements of the matrix to be processed and then accumulating them, there is no need to wait for the vector of the matrix data to be processed to be completely copied, and thus there is no need to wait for the system pipeline to be cleared to start the calculation of the next time step. Only part of the input vector is needed to start the calculation, forming a data pipeline, avoiding pauses, reducing idle hardware resources, and thus improving the utilization of hardware resources. At the same time, according to the scale parameter of the recursive neural network, the element parallel parameters and the vector parallel parameters corresponding to the scale parameter are matched to perform slicing, and the corresponding addition tree tail is configured, so that the slicing strategy can be more flexible and applicable to recursive neural networks of various sizes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a process flow of a matrix data processing method provided by an embodiment of the present invention;

[0035] Figure 1a A schematic diagram of an existing matrix data processing method provided by an embodiment of the present invention;

[0036] Figure 1b A schematic diagram of reasoning of a long short-term memory network provided by an embodiment of the present invention;

[0037] Figure 1c A schematic diagram of a combined weight matrix provided by an embodiment of the present invention;

[0038] Figure 1d A schematic diagram of a process flow of a matrix data processing method provided by an embodiment of the present invention;

[0039] Figure 1e A schematic diagram of a row-wise vector multiplication provided by an embodiment of the present invention;

[0040] Figure 1f A schematic diagram of a column-based vector multiplication provided by an embodiment of the present invention;

[0041] Figure 2 A schematic flow chart of another matrix data processing method provided by an embodiment of the present invention;

[0042] Figure 3 A schematic diagram of a weight vector block provided by an embodiment of the present invention;

[0043] Figure 3a A schematic diagram of an addition tree tail provided by an embodiment of the present invention;

[0044] Figure 3b A schematic diagram of another addition tree tail provided by an embodiment of the present invention;

[0045] Figure 4 A schematic diagram of the structure of a matrix data processing device provided by an embodiment of the present invention;

[0046] Figure 5 A schematic diagram of the structure of an output module provided by an embodiment of the present invention;

[0047] Figure 6 A schematic diagram of the structure of another matrix data processing device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The preferred embodiments of the present invention are described below. A person skilled in the art will be able to implement the embodiments using relevant techniques in the art according to the following description and will be able to better understand the innovations and benefits of the present invention.

[0049] The present invention provides a method for processing matrix data. The purpose of the present invention is achieved through the following technical solutions:

[0050] See also Figure 1 , Figure 1 is a flowchart of a matrix data processing method provided by an embodiment of the present invention, such as Figure 1 As shown, the method is used for a recurrent neural network, and the method comprises the following steps:

[0051] 101. Obtain the matrix data to be processed and the weight matrix data of the recursive neural network.

[0052] The matrix data to be processed and the weight matrix data are both composed of matrix elements, and the matrix data includes column vectors constructed by the matrix elements.

[0053] The matrix data to be processed may be voice matrix data, text matrix data, image matrix data, etc. The voice matrix data may be obtained by encoding voice information into a matrix space, the text matrix data may be obtained by encoding text information into a matrix space, and the image matrix data may be a pixel matrix of the image itself, or may be obtained by encoding the pixel matrix of the image itself into a matrix space.

[0054] The weight matrix data is the weight matrix of the recursive neural network after training. In the processing of the matrix data to be processed, the weight matrix is ​​used to extract the implicit information of the matrix to be processed, so as to obtain the corresponding classification information according to the implicit information.

[0055] The recursive neural network can be deployed in hardware environments such as CPU (central processing unit), GPU (graphics processing unit), FPGA (field programmable gate array), etc. In an embodiment of the present invention, the recursive neural network is preferably deployed in a hardware environment based on FPGA. Compared with the hardware environment of CPU and GPU, the hardware environment of FPGA has the hardware support of logic gates, so the recursive neural network running in the hardware environment of FPGA can have the advantages of low latency and low power consumption.

[0056] The recursive neural network can be a recursive neural network such as a long short-term memory network and a gated recurrent unit (in which the input gate and the forget gate are merged into one update gate). Among them, the recursive neural network described in the embodiment of the present invention is preferably a long short-term memory network. Further, the recursive neural network described in the embodiment of the present invention is preferably a long short-term memory network deployed in the hardware environment of FPGA. It should be noted that the embodiment of the present invention only uses the long short-term memory network as an embodiment to facilitate the explanation of the inventive intent of the present invention, and is also applicable to other forms of recursive neural networks. Here, the long short-term memory network should not be used as a limitation on the scope of the present invention.

[0057] The weight matrix data is also the weight matrix data of the gate in the long short-term memory network. Specifically, the long short-term memory network includes four gates, namely the input gate, the forget gate, the input modulation gate and the output gate. Among them, the input modulation gate can be understood as a sub-part of the input gate, which is used to combine the input tensor with the implicit tensor, so that the corresponding input modulation gate weight matrix represents the weight matrix of the input unit and the implicit unit. Among them, the four gates correspond to each weight matrix respectively. For example, the input gate weight corresponding to the input gate is W i (n), the forget gate weight corresponding to the forget gate is W f (n), the input modulation gate weight corresponding to the input modulation gate is W g (n), the output gate weight corresponding to the output gate is W o (n). Where W i(n), W f (n), W g (n), W o (n) are matrices of the same size.

[0058] In the LSTM network, the hidden state can be calculated through four gates, which can be specifically referred to in the following formula:

[0059] i t =σ(W i [x t ,h t-1 ]+b i ) (1)

[0060] f t =σ(W f [x t ,h t-1 ]+b f ) (2)

[0061] g t =tanh(W f [x t ,h t-1 ]+b u ) (3)

[0062] o t =σ(Wo[x t ,h t-1 ]+bo) (4)

[0063] c t =f t ⊙c t-1 +i t ⊙g t (5)

[0064] h t =o t ⊙tanh(c t ) (6)

[0065] Among them, σ is the normalization function, x t is the current input, h t-1 is the previous implicit state, i t is the calculation result of the input gate, W is the weight matrix of each gate, b is the bias, and f t is the calculation result of the input gate, g t is the calculation result of the input gate, o t is the calculation result of the input gate, c t is the memory cell state, h tis the hidden state of the current input. The above hidden state can be represented by a tensor, that is, a hidden tensor, which can be used as the input of the next time step or the next calculation layer, such as Figure 1b As shown, Figure 1b in,i t , f t , g t , o t The calculation can be called gate calculation (LSTM-Gates), c t ,h t The calculation can be called tail calculation (LSTM-Tail).

[0066] Further, in an embodiment of the present invention, the weight matrix data can be combined by combining the weight matrix data corresponding to the four gates, that is, W i (n), W f (n), W g (n), W o (n) is combined into W(n). Assume that the tensor of weight matrix data corresponding to the four gates is L h *L k , the combined weight matrix data W(n) is a tensor H w *L w , where L h is the number of rows of weight matrix data corresponding to the four gates, L k is the number of columns of the weight matrix data corresponding to the four gates. Similarly, H w is the number of rows of the weight matrix data W(n), L w is the number of columns of the weight matrix data W(n), then H w =4*L h , L w =L k In a possible embodiment, when the tensor size corresponding to the input matrix data is specified, the above L k =L h +L x , L x is the number of rows of matrix data to be processed. At this time, L w =L k =L h +L x .like Figure 1c As shown, in Figure 1c In, W i (0), W f (0), W g (0), W o(0) are the row vectors of the first row of input gate weights, the row vectors of the first row of forget gate weights, the row vectors of the first row of input modulation gate weights, and the row vectors of the first row of output gate weights. They are also the row vectors of the first four rows of the weight matrix data W(n).

[0067] Specifically, the combination of the weight matrix data corresponding to the above-mentioned four gates can be to combine the corresponding row vectors. For example, the row vectors of the first row in the weight matrix data corresponding to the four gates are combined. In the obtained weight matrix data W(n), the row vectors of the first four rows correspond to the row vectors of the first row in the weight matrix data corresponding to the four gates.

[0068] After combining the weight matrix data corresponding to the four gates, a larger tensor of weight matrix data W(n) is obtained. In this way, when optimizing the weight matrix within a time step, it is only necessary to multiply the weight matrix data W(n) by a vector for optimization, instead of optimizing the weight matrices corresponding to the four gates separately through four vectors, which saves the optimization calculation amount and optimization time.

[0069] 102. Extract the weight column vector from the weight matrix data.

[0070] In this step, the above-mentioned weight matrix data is the weight matrix data W(n) in step 101, and the above-mentioned weight column vector can also be called a weight column vector, and each weight column vector can express a column of data in the weight matrix data.

[0071] In this case, the multiplication of the weight matrix data and the matrix data to be processed is performed through row vectors. At this time, it is necessary to extract all the column vectors of the matrix data to be processed to complete the vector multiplication calculation. For example: the tensor of the weight matrix data is n*m, and the tensor of the matrix data to be processed should be j*k, then j=m must be satisfied to perform the vector multiplication calculation. Therefore, it is necessary to extract a complete column of the matrix data to be processed to perform the vector multiplication calculation. Figure 1d As shown, in Figure 1d In the example, Weights Matrix is ​​the weight matrix, W0, W1, W Hw－2 , W Hw－1 are all weight row vectors, and the weight matrix has a total of H w OK, there are L w Column, 0, 1, …, L x-1 is the matrix data to be processed, with a total of L x row, 0, 1, …, L h-1 is the previous implicit tensor, with a total of L h At this time, it is necessary to satisfy L w =L x +Lh To start the vector multiplication calculation, you need to read L x +L h The calculation of vector multiplication can only start after the matrix elements are read. Before the reading is completed, the hardware computing resources are idle.

[0072] 103. Extract the matrix elements to be processed corresponding to the matrix data to be processed and the weight column vector.

[0073] In this step, the above-mentioned matrix data to be processed is extracted in units of matrix elements, rather than in units of vectors. A vector includes multiple matrix elements, thereby shortening the time to start the calculation. Specifically, if the extraction is performed in units of vectors, it is necessary to wait until all matrix elements contained in the vector are read before the calculation starts, while in units of matrix elements, the calculation will start after reading a matrix element, without waiting for the vector of the matrix data to be processed to be completely copied, so that the calculation of the next time step can be started without waiting for the system pipeline to be cleared, and only part of the input vector is needed to start the calculation.

[0074] It should be noted that the above-mentioned matrix elements to be processed are matrix elements in the matrix data to be processed.

[0075] 104. Perform multiplication calculation on the weight column vector and the element of the matrix to be processed to obtain a first processing result.

[0076] In this step, taking the tensor of the weight matrix data as 3*3, the matrix to be processed is 3*1, assuming that the first column in the weight matrix data is the first weight column vector, the first row of the matrix to be processed is the first matrix element, and assuming that the second column in the weight matrix data is the second weight column vector, the third column in the weight matrix data is the third weight column vector, the second row of the matrix to be processed is the second matrix element, and the third row of the matrix to be processed is the third matrix element, then according to step 104, the first weight column can be multiplied with the first matrix element, the second weight column can be multiplied with the second matrix element, and the third weight column can be multiplied with the third matrix element. The tensors of the first weight column vector, the second weight column, and the third weight column are all 3*1, and the tensors of the first matrix element, the second matrix element, and the third matrix element can also be regarded as 1*1, then the corresponding 3*1 is multiplied with 1*1, and the first processing result of three 3*1 tensors is finally obtained. Compared with the direct multiplication of the weight matrix data and the matrix to be processed, that is, the multiplication of 3*3 and 3*1, there is no need to wait for the extraction of the entire 3*1 tensor before calculation. After each 1*1 tensor (matrix element) is extracted, calculation can be performed, making the data closer to streaming and reducing the idle time of hardware resources. Figure 1e As shown, in Figure 1eIn the above example, the column vector of the weight matrix is ​​directly multiplied by a matrix element in the matrix data to be processed. Since the number of columns of the column vector of the weight matrix is ​​1 and the number of rows of the matrix element is 1, the calculation of vector multiplication is satisfied.

[0077] 105. Accumulate the first processing results to obtain a second processing result, and output the second processing result as the processing result of the matrix data to be processed.

[0078] In this step, taking the tensor of the weight matrix data in step 104 as 3*3 and the matrix to be processed as 3*1, the first processing result obtained is three 3*1 tensors. The three 3*1 tensors are accumulated to obtain a second processing result of a 3*1 tensor. Compared with the direct multiplication of the weight matrix data and the matrix to be processed, that is, the multiplication of 3*3 and 3*1, the result of the multiplication of 3*3 and 3*1 is a 3*1 tensor, which can also be called an implicit tensor or an implicit state. The multiplication of the weight column vector and the matrix element, that is, the multiplication of 3*1 and 1*1, is finally accumulated to obtain a 3*1 tensor, which can also be called an implicit tensor or an implicit state. After each 1*1 tensor (matrix element) is extracted by the multiplication of the weight column vector and the matrix element, it can be calculated, making the data closer to streaming and reducing the idle time of hardware resources. Figure 1e as well as Figure 1f As shown, in Figure 1f As can be seen, there is no need to wait for the implicit tensor h t All data is calculated like an assembly line without any stagnation or waiting.

[0079] In this embodiment, the matrix data to be processed and the weight matrix data of the recursive neural network are obtained, and the matrix data to be processed and the weight matrix data are both composed of matrix elements, and the matrix data includes a column vector constructed by the matrix elements; the weight column vector in the weight matrix data is extracted; the matrix elements to be processed corresponding to the matrix data to be processed and the weight column vector are extracted; the weight column vector is multiplied by the matrix elements to be processed to obtain a first processing result; the first processing result is accumulated to obtain a second processing result, and the second processing result is output as the processing result of the matrix data to be processed. By multiplying the column vector of the weight matrix data with the matrix elements of the matrix to be processed and then accumulating, there is no need to wait for the vector of the matrix data to be processed to be completely copied, so that the calculation of the next time step can be started without waiting for the system pipeline to be cleared, and only part of the input vector is needed to start the calculation, forming a data pipeline, avoiding pauses, reducing the idleness of hardware resources, and thus improving the utilization of hardware resources.

[0080] See also Figure 2 , Figure 2FIG. 1 is a flowchart of another matrix data processing method according to an embodiment of the present invention. Figure 2 As shown, the method is used for recurrent neural networks, in Figure 1 Based on the embodiment, the method comprises the following steps:

[0081] 201. Obtain the matrix data to be processed and the weight matrix data of the recursive neural network.

[0082] The matrix data to be processed and the weight matrix data are both composed of matrix elements, and the matrix data includes column vectors constructed by the matrix elements.

[0083] The matrix data to be processed may be voice matrix data, text matrix data, image matrix data, etc. The voice matrix data may be obtained by encoding voice information into a matrix space, the text matrix data may be obtained by encoding text information into a matrix space, and the image matrix data may be a pixel matrix of the image itself, or may be obtained by encoding the pixel matrix of the image itself into a matrix space.

[0084] The weight matrix data is the weight matrix of the recursive neural network after training. In the processing of the matrix data to be processed, the weight matrix is ​​used to extract the implicit information of the matrix to be processed, so as to obtain the corresponding classification information according to the implicit information.

[0085] 202. According to a scale parameter of the recursive neural network, match an element parallel parameter and a vector parallel parameter corresponding to the scale parameter.

[0086] In an embodiment of the present invention, the scale parameter of the above-mentioned recursive neural network can be determined by the number of processing units NPE.

[0087] In this step, the available parallelism can be further exploited for the element parallelism parameter (EP) and the vector parallelism parameter (VP) so that the number of computation cycles in the processing process is greater than the delay. It should be noted that the element parallelism parameter represents the number of parallel processing of matrix elements, and the vector parallelism parameter represents the number of rows of a column-type vector.

[0088] Furthermore, the vector parallel parameters are constrained by the weight matrix data and the element parallel parameters. Specifically, the vector parallel parameters can be obtained by first obtaining the number of processing units (NPE); obtaining the number of vector rows of the weight matrix in the weight matrix data; constraining the vector parallel parameters according to the ratio of the number of processing units to the element parallel parameters and the weight matrix width, and obtaining the element parallel parameters and the vector parallel parameters through a greedy algorithm search. Specifically, the above constraints can be:

[0089] VP≤H w =4*Lh (7)

[0090] VP≤NPE / EP (8)

[0091] Through the above two formulas, the element parallel parameter is taken as a variable, starting from the element parallel parameter EP=1, a greedy algorithm is performed to search for the best vector parallel parameter and element parallel parameter.

[0092] It can be seen that when EP is small, since VP is constrained by formula (7), the number of processing cycles is high, making the effective processing unit PE (each processing unit has EP elements to process) less than the number of processing units NPE, resulting in serious underutilization. When EP increases, the number of processing cycles decreases until EP reaches these optimal points. When EP is greater than the optimal point, the processing cycle gradually increases. For example, according to the preset design space exploration results, when NPE = 16382, the EP value of the optimal configuration is between 4 and 16; when NPE = 65536, the EP value is between 16 and 64. At these optimal positions, high parallelism can be achieved, thereby improving system throughput.

[0093] Optionally, the scale parameters of the above recursive neural network may also include: the number of processing units and the vector dimension. In this optional embodiment, the element parallelism parameters and vector parallelism parameters corresponding to the number of processing units and the vector dimension may be matched according to the number of processing units and the vector dimension. In this way, the element parallelism parameters and vector parallelism parameters that are more suitable for the current recursive neural network scale may be obtained for slicing and blocking.

[0094] For example, for a given vector size, better performance and utilization can be achieved by adjusting the (EP, VP) design parameters. For example, for 512-dimensional vectors, the performance under (EP, VP) of (32, 2048) is better than that under (EP, VP) of (16, 4096), while for 1024-dimensional vectors, the performance under (EP, VP) of (16, 4096) is better than that under (EP, VP) of (32, 2048). When running RNN models of different sizes, different choices of EP and VP will affect the hardware utilization and performance of the architecture.

[0095] It should be noted that each PE is equivalent to a full-pipeline multiplier, which is used for column-based multiplication calculations. After passing through the PE, it will enter the accumulator for addition calculations.

[0096] 203. Divide the weight matrix data into blocks according to the element parallel parameters and the vector parallel parameters to obtain weight vector blocks.

[0097] In this step, the weight vector block can be Figure 3As shown, it should be noted that the element parallel parameters in the weight vector block and the element parallel parameters in the matrix data to be processed have the same number of matrix elements, that is, they have the same EP. The weight vector block obtained by the element parallel parameters and the vector parallel parameters can improve the inference throughput of the recursive neural network.

[0098] In an embodiment of the present invention, since the above-mentioned element parallel parameters and vector parallel parameters are obtained according to the scale parameters of the recursive neural network, the element parallel parameters and vector parallel parameters can be flexibly configured according to the scale of the recursive neural network, and then the weight matrix data can be flexibly divided into blocks, and the obtained weight vector blocks also conform to the scale of the current recursive neural network, further improving the utilization rate of the hardware.

[0099] 204. Perform multiplication calculation on the weight vector block and the matrix element to be processed to obtain a first processing result.

[0100] The weight vector blocks obtained by dividing the weight matrix data into blocks are multiplied by the parallel parameters of the elements in the matrix to be processed relative to the matrix elements to obtain the first processing result. This can improve the reasoning throughput of the recursive neural network.

[0101] 205. Configure an addition tree tail according to an element parallel parameter, accumulate the first processing result through an addition tree including the addition tree tail to obtain a second processing result, and output the second processing result as a processing result of the matrix data to be processed.

[0102] In the embodiment of the present invention, in order to support various EPs and VPs, an adder tree tail (CAT) can be configured to reduce the configuration of adders. Corresponding to various numbers of EPs, the number of levels of the adder tree can be changed accordingly. For example, if the fixed structure of the adder tree is for large EPs, the results of the last few levels of the adder tree can be used to directly update the accumulator with a small EP without inputting the next level of adders, thereby reducing the configuration of adders.

[0103] Furthermore, the tail of the addition tree is arranged behind the addition tree; and the tail of the addition tree is arranged in parallel.

[0104] In the CAT architecture with an adder tree tail, the adders at the end of the adder tree can be reused as the required accumulators without the need for additional adder components. For example, a CAT with N inputs (CAT-N) can be configured to update 1 to N accumulators when data reaches the last log2(N) level of the adder tree. Figure 3a Therefore, all three addition trees have 4 inputs, the addition tree tail is CAT-4, and the result of the addition tree is used to update accumulators 1 to 4. Optionally, the structural details of another addition tree tail CAT-4 are as follows Figure 3b shown.

[0105] In the design of large-scale recursive neural networks, according to the design exploration of the optimal system throughput, since the number of EPs is {16, 32, 64}, it is sufficient to use the addition tree tail CAT-4. Of course, it is also possible to cascade the addition tree tail CAT-N in the addition tree tail CAT-N, for example, cascading two CAT-2s to get a CAT-4.

[0106] In an embodiment of the present invention, dequantization (inverse quantization) may also be performed after the adapter. Since dequantization and quantization require a 32-bit multiplier unit and an adder, the total hardware cost of linear quantization at the end of the addition tree is higher than the hardware cost of 16-bit fixed-point quantization. Therefore, the end of the addition tree may not perform linear quantization. Instead, dequantization is performed after the adapter, and the quantized value is dequantized into a fixed-point value. The output of the end of the addition tree will generate an output vector that does not require inverse quantization, avoiding the addition of inverse quantization hardware at the end of the addition tree, thereby reducing the total hardware cost.

[0107] Furthermore, a balance calculation is performed on the first processing result to balance the parallelism of the weight vector blocks.

[0108] Specifically, in the architecture of the recursive neural network, there are multipliers and accumulators, and the accumulator is connected after the multiplier, wherein the multiplier is used for the calculation of vector multiplication, specifically for the calculation of vector multiplication between the above-mentioned column vector and the matrix elements, or for the calculation of vector multiplication between the weight vector block and the elements in the matrix to be processed and the parallel parameters of the elements in the matrix to be processed. The above-mentioned accumulator is used to accumulate the first processing result.

[0109] Optionally, a balanced adder tree may be set between the multiplier and the accumulator to perform a balance calculation on the first processing result, specifically, to balance the parallelism of element parallel parameters and vector parallel parameters to further increase the inference throughput of the recursive neural network.

[0110] In this embodiment, the matrix data to be processed and the weight matrix data of the recursive neural network are obtained, and the matrix data to be processed and the weight matrix data are both composed of matrix elements, and the matrix data include column vectors constructed by the matrix elements; according to the scale parameter of the recursive neural network, the element parallel parameters and the vector parallel parameters corresponding to the scale parameter are matched; according to the element parallel parameters and the vector parallel parameters, the weight matrix data are divided into blocks to obtain weight vector blocks; the weight vector blocks are multiplied by the matrix elements to be processed to obtain a first processing result; according to the element parallel parameters, the addition tree tail is configured, the first processing result is accumulated through the addition tree including the addition tree tail to obtain a second processing result, and the second processing result is output as the processing result of the matrix data to be processed. By multiplying the column vector of the weight matrix data with the matrix elements of the matrix to be processed and then accumulating them, there is no need to wait for the vector of the matrix data to be processed to be completely copied, and thus there is no need to wait for the system pipeline to be cleared to start the calculation of the next time step. Only part of the input vector is needed to start the calculation, forming a data pipeline, avoiding pauses, reducing idle hardware resources, and thus improving the utilization of hardware resources. At the same time, according to the scale parameter of the recursive neural network, the element parallel parameters and the vector parallel parameters corresponding to the scale parameter are matched to perform slicing, and the corresponding addition tree tail is configured, so that the slicing strategy can be more flexible and applicable to recursive neural networks of various sizes.

[0111] See also Figure 4 , Figure 4 A matrix data processing device provided by an embodiment of the present invention is used for a recursive neural network, and is characterized in that the device comprises:

[0112] An acquisition module 401 is used to acquire the to-be-processed matrix data and the weight matrix data of the recursive neural network, wherein the to-be-processed matrix data and the weight matrix data are both composed of matrix elements, and the matrix data includes column vectors constructed by the matrix elements;

[0113] A matching module 402, configured to match element parallel parameters and vector parallel parameters corresponding to the scale parameter of the recursive neural network according to the scale parameter of the recursive neural network;

[0114] A processing module 403 is used to divide the weight matrix data into blocks according to the element parallel parameter and the vector parallel parameter to obtain weight vector blocks;

[0115] A calculation module 404 is used to perform multiplication calculation on the weight vector block and the matrix element to be processed to obtain a first processing result;

[0116] The output module 405 is used to configure the addition tree tail according to the element parallel parameter, accumulate the first processing result through the addition tree including the addition tree tail to obtain a second processing result, and output the second processing result as the processing result of the matrix data to be processed.

[0117] Optional, such as Figure 5 As shown, the output module 405 includes:

[0118] A first configuration unit 4051 is used to configure the addition tree tail behind the addition tree;

[0119] The second configuration unit 4052 is used to configure the parallelism of the addition tree tail.

[0120] Optionally, the scale parameters of the recursive neural network include: the number of processing units and the vector dimension, and the matching module 402 is further used to match the element parallel parameters and vector parallel parameters corresponding to the number of processing units and the vector dimension according to the number of processing units and the vector dimension.

[0121] Optional, such as Figure 6 As shown, the device also includes:

[0122] The balancing module 406 is used to perform a balance calculation on the first processing result to balance the parallelism of the weight vector blocks.

[0123] In an embodiment of the present invention, by multiplying the column vector of the weight matrix data with the matrix elements of the matrix to be processed and then accumulating them, there is no need to wait for the vector of the matrix data to be processed to be completely copied, so that the calculation of the next time step can be started without waiting for the system pipeline to be cleared. Only part of the input vector is needed to start the calculation, forming a data pipeline, avoiding pauses, reducing idle hardware resources, and thus improving the utilization of hardware resources. At the same time, according to the scale parameter of the recursive neural network, the element parallel parameters and the vector parallel parameters corresponding to the scale parameter are matched to perform slicing, and the corresponding addition tree tail is configured, so that the slicing strategy can be more flexible and applicable to recursive neural networks of various sizes.

[0124] An embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the matrix data processing method provided by the embodiment of the present invention when executing the computer program.

[0125] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the matrix data processing method provided by the embodiment of the present invention are implemented.

[0126] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0127] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0128] In the several embodiments provided by the present invention, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative.

[0129] In addition, the processors and chips in various embodiments of the present invention may be integrated into a processing unit, or may exist physically separately, or two or more hardware may be integrated into one unit. A computer-readable storage medium or a computer-readable program may be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution may be embodied in the form of a software product, which is stored in a memory and includes several instructions for enabling a computer device (which may be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in various embodiments of the present invention. The aforementioned memory includes: various media that can store program codes, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0130] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk, etc.

[0131] The above content is a further detailed description of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific embodiments of the present invention are limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A method for processing matrix data for a recursive neural network, characterized in that: The method comprises: Obtaining matrix data to be processed and weight matrix data of the recursive neural network, wherein both the matrix data to be processed and the weight matrix data are composed of matrix elements, the matrix data includes column vectors constructed by the matrix elements, and the matrix data to be processed is at least one of speech matrix data, text matrix data, and image matrix data; According to a scale parameter of the recursive neural network, matching an element parallel parameter and a vector parallel parameter corresponding to the scale parameter; Divide the weight matrix data into blocks according to the element parallel parameter and the vector parallel parameter to obtain weight vector blocks; Perform multiplication calculation on the weight vector block and the matrix element to be processed to obtain a first processing result; The addition tree tail is configured according to the element parallel parameter, the first processing result is accumulated through the addition tree including the addition tree tail to obtain a second processing result, and the second processing result is output as the processing result of the matrix data to be processed.

2. The method for processing matrix data according to claim 1, characterized in that: The step of configuring the addition tree tail according to the element parallel parameters includes: Arrange the addition tree tail behind the addition tree; Configure parallelism of the addition tree tail.

3. The method for processing matrix data according to claim 1, characterized in that: The scale parameter of the recursive neural network includes: the number of processing units and the vector dimension, and the matching of the element parallelism parameter and the vector parallelism parameter corresponding to the scale parameter according to the scale parameter of the recursive neural network includes: According to the number of processing units and the vector dimension, element parallelism parameters and vector parallelism parameters corresponding to the number of processing units and the vector dimension are matched.

4. The method for processing matrix data according to claim 1, characterized in that: Before accumulating the first processing result through the addition tree including the addition tree tail to obtain the second processing result, the method further includes: A balance calculation is performed on the first processing result to balance the parallelism of the weight vector blocks.

5. A matrix data processing device for a recursive neural network, characterized in that: The device comprises: An acquisition module, used for acquiring matrix data to be processed and weight matrix data of the recursive neural network, wherein the matrix data to be processed and the weight matrix data are both composed of matrix elements, the matrix data includes column vectors constructed by the matrix elements, and the matrix data to be processed is at least one of speech matrix data, text matrix data and image matrix data; A matching module, configured to match element parallel parameters and vector parallel parameters corresponding to the scale parameter of the recursive neural network according to the scale parameter of the recursive neural network; A processing module, used for dividing the weight matrix data into blocks according to the element parallel parameters and the vector parallel parameters to obtain weight vector blocks; A calculation module, used for performing multiplication calculation on the weight vector block and the matrix element to be processed to obtain a first processing result; An output module is used to configure an addition tree tail according to an element parallel parameter, accumulate the first processing result through an addition tree including the addition tree tail to obtain a second processing result, and output the second processing result as a processing result of the matrix data to be processed.

6. The matrix data processing device according to claim 5, characterized in that: The output module comprises: A first configuration unit, configured to configure the addition tree tail behind the addition tree; The second configuration unit is used to configure the parallelism of the addition tree tail.

7. The matrix data processing device according to claim 5, characterized in that: The scale parameters of the recursive neural network include: the number of processing units and the vector dimension. The matching module is also used to match the element parallel parameters and vector parallel parameters corresponding to the number of processing units and the vector dimension according to the number of processing units and the vector dimension.

8. The matrix data processing device according to claim 5, characterized in that: The device also includes: A balancing module is used to perform a balance calculation on the first processing result to balance the parallelism of the weight vector blocks.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps in the matrix data processing method according to any one of claims 1 to 4 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the matrix data processing method according to any one of claims 1 to 4 are implemented.

Images