Data processing method and device, electronic equipment and storage medium

Abstract

The present disclosure provides a data processing method and device, electronic equipment and storage medium, wherein the method comprises: obtaining input features and weight parameters of a full connection layer; splitting the input features into a plurality of input matrices, and splitting the weight parameters into a plurality of weight matrices correspondingly, at least one input matrix has more than one feature in the row direction and / or more than one feature in the column direction; inputting the plurality of input matrices and the plurality of weight matrices into a PE array for convolution operation to obtain output features of the full connection layer. In the splitting process, a part of the data input by the full connection layer in one dimension can be split to the row / column to participate in subsequent operation, which can occupy more calculation units when starting the PE array each time, thereby improving the utilization rate of the PE array.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to a data processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the development of artificial intelligence, neural network models such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN) and Deep Neural Networks (DNN) have been applied in various fields, such as speech recognition, image classification, object detection, etc.

[0003] In actual applications, the fully connected layer is an indispensable part of the neural network model, which is used to output the operation results of the previous layer of neurons to the inputs of all neurons of the next layer, so that the next layer can continue to operate. As the main computing unit in the neural network model, the Processing Element (PE) array provides a scheme for convolution calculation of the fully connected layer.

[0004] However, due to the computing characteristics of the PE array itself, the utilization rate of the PE array is not high during the calculation of the fully connected layer. SUMMARY

[0005] The embodiments of the present disclosure at least provide a data processing method, device, electronic equipment and storage medium to improve the utilization rate of the PE array during the full connection calculation by splitting the data of the fully connected layer.

[0006] In a first aspect, the embodiments of the present disclosure provide a data processing method, comprising:

[0007] obtaining input features and weight parameters of a fully connected layer;

[0008] splitting the input features into a plurality of input matrices, and splitting the weight parameters into a plurality of weight matrices, at least one of the input matrices has more than one feature in the row direction and / or more than one feature in the column direction;

[0009] inputting the plurality of input matrices and the plurality of weight matrices into a PE array for convolution operation to obtain output features of the fully connected layer.

[0010] In a possible implementation, the row and column distributions of the plurality of input matrices are consistent.

[0011] In a possible implementation, the number of the input matrices is greater than or equal to the number of rows of the PE array.

[0012] In a possible implementation, the number of the input matrices is an integer multiple of the number of rows of the PE array.

[0013] In a possible implementation, the number of features of the input matrix is less than or equal to a calculation threshold of a calculation unit of the PE array.

[0014] In a possible implementation, the number of features of the input matrix in the row direction is less than or equal to a calculation threshold of a calculation unit of the PE array in the row direction.

[0015] In a possible implementation, the number of features of the input matrix in the column direction is less than or equal to a calculation threshold of a calculation unit of the PE array in the column direction.

[0016] In a possible implementation, the input feature of the fully connected layer is a one-dimensional vector, and the number of features of the weight parameter in the column direction is consistent with the number of features of the one-dimensional vector.

[0017] In a possible implementation, the inputting of the plurality of input matrices and the plurality of weight matrices into the PE array for convolution operation includes:

[0018] For each input matrix and the corresponding weight matrix, a calculation unit of the PE array is used to perform a point multiplication operation on the input matrix and the corresponding weight matrix to obtain an operation result.

[0019] The operation results of each input matrix and the corresponding weight matrix are summarized to obtain the output feature of the fully connected layer.

[0020] In a possible implementation, the summarizing of the operation results of each input matrix and the corresponding weight matrix includes:

[0021] The operation results of the calculation units in the same column in the PE array are superimposed to obtain the output feature of the fully connected layer.

[0022] In a second aspect, the embodiments of the present disclosure further provide a data processing apparatus, including:

[0023] An obtaining module is configured to obtain an input feature and a weight parameter of a fully connected layer.

[0024] A splitting module is configured to split the input feature into a plurality of input matrices, and split the weight parameter into a plurality of weight matrices correspondingly, at least one of the input matrices has a number of features greater than 1 in the row direction and / or a number of features greater than 1 in the column direction.

[0025] The operation module is configured to input the plurality of input matrices and the plurality of weight matrices into the PE array to perform convolution operation, and obtain the output feature of the fully connected layer.

[0026] In a third aspect, the embodiments of the present disclosure further provide an electronic device, including a processor, a memory and a bus, the memory stores machine readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the machine readable instructions are executed by the processor to perform the data processing method in any one of the first aspect and various embodiments thereof.

[0027] In a fourth aspect, the embodiments of the present disclosure further provide a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to perform the data processing method in any one of the first aspect and various embodiments thereof.

[0028] By using the above data processing method, device, electronic device and storage medium, in the case of obtaining the input feature and weight parameter of the fully connected layer, the input feature can be split into a plurality of input matrices, and the weight parameter is correspondingly split into a plurality of weight matrices, so that after the plurality of input matrices and the plurality of weight matrices are input into the PE array for convolution calculation, the output feature of the fully connected layer can be obtained. In the splitting process, whether the number of features of the input matrix obtained by splitting is greater than 1 in the row direction or the number of features is greater than 1 in the column direction, the data in one dimension of the fully connected layer input can be split to the row / column to participate in the subsequent operation, which will make more calculation units can be occupied when starting the PE array each time, thereby improving the utilization rate of the PE array.

[0029] Other advantages of the present disclosure will be described in more detail in conjunction with the following description and drawings.

[0030] It should be understood that the above description is only a summary of the technical solutions of the present disclosure, so as to enable a clearer understanding of the technical means of the present disclosure, so that the content of the specification can be implemented. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the following specific embodiments of the present disclosure are exemplified. BRIEF DESCRIPTION OF DRAWINGS

[0031] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort. Furthermore, the same reference numerals denote the same components throughout the drawings. In the drawings:

[0032] Figure 1 A flowchart of a data processing method provided by an embodiment of this disclosure is shown;

[0033] Figure 2 A schematic diagram of a PE array provided in an embodiment of this disclosure is shown;

[0034] Figure 3 A schematic diagram of a data processing apparatus provided in an embodiment of this disclosure is shown;

[0035] Figure 4 A schematic diagram of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0036] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0037] In the description of embodiments disclosed herein, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of features, numbers, steps, behaviors, components, portions or combinations thereof disclosed herein, and are not intended to exclude the possibility of the presence of one or more other features, numbers, steps, behaviors, components, portions or combinations thereof.

[0038] Unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In this article, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0039] The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more.

[0040] Research has shown that the PE array, as the main computing unit in an artificial intelligence (AI) chip, can perform operations between the input feature matrix and the weight matrix, which are typically 4-dimensional matrices. To further understand this operational process, specific formulas will be used to describe it below.

[0041] Assume that Ifmap(input feature): the input feature matrix, represented by N, C, H, W from high to low dimensions; weight: the weights, represented by M, C, R, S from high to low dimensions; and Ofmap(output feature): the output feature matrix, which is the result of the convolution of ifmap and weight, represented by N, M, E, F from high to low dimensions. The specific convolution calculation formula is as follows:

[0042]

[0043] Based on the above convolution formula, it is known that each point of the convolution output requires multiplication and addition operations in the C, R, and S directions. To implement operations on fully connected layers, a scheme is provided to compute the fully connected layers using a convolutional approach. In fully connected layer computation, R=1, S=1, E=1, F=1, and the formula can be simplified to:

[0044]

[0045] As can be seen from the above formula, when the fully connected layer is calculated in the manner of convolution, it only needs to perform multiplication and addition in the C direction.

[0046] In computation based on the current mainstream PE array, the column direction (PE_HEIGHT) is responsible for performing multiplication and addition in the C direction. However, since the computation of fully connected layers only utilizes multiplication and addition in the C direction, multiplication and addition in the R and S directions cannot be effectively utilized.

[0047] In addition, during each PE array startup (corresponding to one pass), only PE_HEIGHT (corresponding to the number of PE rows) C-direction multiply-accumulate operations can be processed. This requires C / PE_HEIGHT pass calculations, meaning C / PE_HEIGHT restarts are needed. In a single pass, the R and S-direction multiply-accumulate operations are only started once; that is, one loop can complete the calculation of one pass. Assuming the configuration instruction issuance time for one pass is n loop times, since the calculation time is only one loop time, the actual PE array utilization is only 1 / (n+1). Therefore, the PE array has extremely low utilization during fully connected operations.

[0048] To at least partially solve one or more of the above-mentioned problems and other potential problems, this disclosure provides at least one data processing scheme that splits the input data of the fully connected layer before performing operations. This scheme allows the accumulation in the C direction to be performed simultaneously in the R and S directions, which can significantly improve the utilization rate of the PE array within one pass of operation and effectively save the number of instructions generated, thereby improving the computational efficiency of the PE array.

[0049] To facilitate understanding of this embodiment, a data processing method disclosed in this disclosure will first be described in detail. The data processing method provided in this disclosure is generally executed by an electronic device with a certain computing capability. This electronic device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a cellular phone, a computing device, an in-vehicle device, a wearable device, etc. In some possible implementations, this data processing method can be implemented by a processor calling computer-readable instructions stored in memory.

[0050] See Figure 1 The diagram shows a flowchart of a data processing method provided in an embodiment of this disclosure. The method includes steps S101 to S103, wherein:

[0051] S101: Obtain the input features and weight parameters of the fully connected layer;

[0052] S102: The input features are split into multiple input matrices, and the weight parameters are correspondingly split into multiple weight matrices, wherein at least one of the input matrices has a greater than 1 number of features in the row direction and / or a greater than 1 number of features in the column direction;

[0053] S103: Input the multiple input matrices and the multiple weight matrices into the PE array for convolution operation to obtain the output features of the fully connected layer.

[0054] To facilitate understanding of the data processing method provided in this disclosure, the application scenarios of the method will be described in detail below. The data processing method in this disclosure can be applied to any relevant field that requires the participation of fully connected layers in computation. For example, it can be used for speech recognition tasks based on fully connected layers, or for object detection tasks based on fully connected layers, and can also be used in various other application fields, without specific limitations.

[0055] Considering that in the process of using a PE array to implement fully connected layer computation, only a small number of computing units may participate in the actual operation, that is, the utilization rate of the PE array is low. Based on this, the embodiments of this disclosure provide a scheme that first splits the data in the row / column direction, and then performs operations based on the split data. The split data can be well adapted to the PE array, which significantly improves the utilization rate of the PE array in the row / column direction.

[0056] Before splitting the data, let's first briefly explain the structure of the PE array.

[0057] like Figure 2 As shown, this is an example of a 6x6 PE array, which has 6 computational units in each row and column direction, for a total of 36 computational units. For each computational unit, an operation can be performed on the corresponding input matrix obtained by the split. For example, the first computational unit in the first row and first column and the 19th computational unit in the fourth row and first column can both perform the operation on the 20x4 input matrix and the corresponding weight matrix as shown in the figure.

[0058] The input features of the fully connected layer can be a one-dimensional vector [a1, a2, ..., a1280]. This one-dimensional vector can be represented, for example, as NCHW = 1 * 1280 * 1 * 1, meaning it has 1280 feature values ​​in the C dimension. To better adapt to the structural design of the PE array, this one-dimensional vector can be split. For example, only a portion of the 1280 feature values ​​in the C dimension can be split into the H dimension, only a portion into the W dimension, or both the H and W dimensions can be split simultaneously, thus satisfying different splitting requirements as much as possible.

[0059] Here, for the input matrix obtained by splitting in the row direction, the number of features in the row direction will be greater than 1, and for the input matrix obtained by splitting in the column direction, the number of features in the column direction will be greater than 1.

[0060] Assuming the PE array is composed of 16*16 computational units, it can be fully partitioned along dimension C, meaning the number of input matrices can be 16. Taking NCHW = 1*1280*1*1 as an example, splitting it into two sets of input matrices corresponds to two sets of input matrices: NCHW = 1*16*20*2. Each set has 16 input matrices, with 20 eigenvalues ​​extracted along the rows and 2 eigenvalues ​​along the columns. Subsequent operations can then be performed on these eigenvalues. Again, using NCHW = 1*1280*1*1 as an example, splitting it into one set of input matrices corresponds to NCHW = 1*16*20*4. In this case, there are 16 input matrices, with 20 eigenvalues ​​extracted along the rows and 4 eigenvalues ​​along the columns.

[0061] It should be noted that during feature splitting, the splitting can be performed according to the original C-dimensional arrangement. Taking the input matrix NCHW = 1*16*20*4 as an example, the 16 feature values ​​can be kept in the C-dimensional, and the subsequent 20 feature values ​​and the next 4 feature values ​​can be split into the H-dimensional and W-dimensional respectively. In this way, the number of features in the C-dimensional corresponds to the number of rows in the PE array. The 16 20*4 input matrices can be input into the computing units of the PE array to participate in the subsequent convolution operation.

[0062] During the process of splitting the input matrix, the corresponding weight parameters involved in the calculation also need to be split. The splitting method is the same as that for the input matrix, and will not be repeated here.

[0063] It should be noted that when the input feature of the fully connected layer is a one-dimensional vector, the number of features of the weight parameter in the column direction is the same as the number of features of the one-dimensional vector. Here, we still take NCHW = 1*1280*1*1 as an example, which corresponds to 1280 feature values ​​in the C dimension. The weight parameter also has 1280 weight values ​​in the column direction. At this time, a similar split can be performed.

[0064] For the split input matrix and weight matrix, they can be input into the PE array for convolution operation to obtain the output features of the fully connected layer. In this embodiment, the convolution operation can be performed according to the following steps:

[0065] Step 201: For each input matrix and its corresponding weight matrix, use a computing unit of the PE array to perform a dot product operation on the input matrix and the corresponding weight matrix to obtain the operation result;

[0066] Step 202: Summarize the calculation results of each input matrix and its corresponding weight matrix to obtain the output features of the fully connected layer.

[0067] Here, for each input matrix and its corresponding weight matrix forming a pairing matrix, a computing unit of the PE array can be used to perform a dot product operation on the input matrix and the corresponding weight matrix to obtain the operation result. In this way, by summing up the operation results obtained from each pairing matrix, the output features of the entire fully connected layer can be obtained.

[0068] When multiple input matrices are split into full rows according to the PE array, a single PE array startup can determine the operation results of each paired matrix. By superimposing the operation results, the output features of the fully connected layer can be determined.

[0069] It can be seen that the above superposition process is to superimpose the calculation results obtained by the calculation units in the same column of the PE array.

[0070] To better understand the dot product operation in the above convolution process, the following formula can be used as a reference.

[0071]

[0072] As shown in the above equation, after splitting the input feature Ifmap, moving the multiplication and addition in the C direction to the R and S directions effectively utilizes the multiplication and addition in the C, R, and S directions. Compared to related technologies that require C / PE_HEIGHT passes to complete the calculation, this only requires C / (R*S) / PE_HEIGHT passes, which is 1 / (R*S) of the previous method, and the corresponding number of instructions is also only 1 / (R*S).

[0073] In addition, assuming the PE array requires n loops to wait for configuration commands in one pass, the duty cycle of the PE array before optimization is only 1 / n. After optimization, one pass requires R*S loops to complete the calculation. The duty cycle of the optimized PE array is R*S / n, which is an increase of R*S times. This improves the utilization rate of the PE array.

[0074] The data processing method provided in this disclosure can be split in various ways, taking into account factors such as different splitting directions, different splitting orders, and different numbers of splits. For example, it can be split only in the row direction, or only in the column direction, or split in both the row and column directions. Furthermore, it can be split into fewer features or more features. Additionally, it can be split sequentially or in other splitting orders, without any specific limitations.

[0075] If the number of input matrices after splitting / the number of rows in the PE array is less than the number of rows in the initial input feature / the number of rows in the PE array, then the number of passes required to start the PE array is reduced, thus improving the computational efficiency of the PE array. For example, if the input feature is represented as NCHW = 1*1280*1*1, and the PE array is an array of 16*16 computational units, the initial number of rows in the input feature / PE array is 80, requiring 80 PE array starts. After splitting it into 1264 matrices, including 1263 1*1 matrices and 1 1*16, 2*8, 2*4, or 16*1 matrix, the number of input matrices after splitting / the number of rows in the PE array is 79, requiring 79 PE array starts, thus reducing the computation time compared to before splitting.

[0076] To ensure the most even distribution of computing resources, in practical applications, the multiple input matrices can be matrices with consistent row and column distribution, which facilitates the splitting process. Secondly, different input matrices can output the computation results almost simultaneously, thereby ensuring that the output features of the fully connected layer are obtained more quickly to support the computation of the fully connected layer.

[0077] In practical applications, the number of input matrices is greater than or equal to the number of rows in the PE array. This ensures that the row computational resources of the PE array are fully utilized each time it is started. For example, for the input feature NCHW = 1*1280*1*1, with 16 rows in the PE array, it can be split into 16 20*4 input matrices, thus ensuring that the PE array can fully utilize the 16 rows of computational resources for operation each time it is started.

[0078] In some special cases, the number of input matrices can be less than the number of rows of the PE array; no specific restrictions are imposed here.

[0079] To further ensure that the row computation resources of the PE array are fully utilized in each startup, the number of input matrices must be an integer multiple of the number of rows in the PE array. For example, for an input feature of NCHW = 1*1280*1*1, with a PE array of 16 rows, it can be split into 64 5*4 input matrices, thus completing the entire operation in 4 startups. If the splitting does not meet the above requirement of being an integer multiple, there may be a situation where the PE array cannot be fully utilized in a certain startup, which will have a certain impact on the utilization rate of the PE array.

[0080] In this embodiment of the disclosure, the number of features of the input matrix is ​​less than or equal to the computation threshold of the computing unit of the PE array. This is mainly because the more input features there are, the more computing resources are required to a certain extent, and conversely, the fewer input features there are, the less computing resources are required to a certain extent. The above constraint is implemented in order to ensure that the operation is completed within the computing unit of the PE array as much as possible, thereby ensuring that more accurate calculations can be provided subsequently.

[0081] In practical applications, the number of features of the input matrix in the row direction is less than or equal to the calculation threshold of the computing unit of the PE array in the row direction; the number of features of the input matrix in the column direction is less than or equal to the calculation threshold of the computing unit of the PE array in the column direction.

[0082] Taking an input matrix with 20 features in the row direction as an example, the computation threshold of the PE array's computational units in the row direction needs to be greater than 20 to meet the computational requirements.

[0083] To further understand the data processing method provided in the embodiments of this disclosure, a specific example will be used for illustration below.

[0084] Assume that the input feature Ifmap in the fully connected layer is represented as NCHW = 1*1280*1*1, the output feature Ofmap is represented as MEF = 1000*1*1, and the weight parameter RS ​​= 1*1.

[0085] According to the original splitting method, NCHW = 1*16*1*1 and MEF = 32*1*1 in one pass. The calculation time of one pass is only 1 clock cycle. However, at this time, 1280 / 16 = 80 passes are needed. At this time, instruction parsing will become the bottleneck of calculation.

[0086] After splitting the data using the data processing method provided in this embodiment, the original Ifmap and weight can be determined as NCHW = 1*16*20*4 and RS = 20*4, respectively. In one pass, NCHW = 1*16*20*4 and MEF = 32*1*1, and the calculation time is 20*4 = 80 cycles. This allows full utilization of the computing unit for calculation, reducing the number of instructions, i.e., only one pass is needed.

[0087] The test was performed using the last fully connected layer in mobilenet_v2_1.0.224: NCHW = 1*1280*1*1, MEF = 1001*1*1.

[0088] Here, the total computation time of the original splitting method test instructions is 652938ns. However, after optimization using the data processing method provided in this embodiment, NCHW is split into 1*16*20*4 and MCRS is split into 1001*16*20*4, and the total computation time is only 150915ns, which significantly improves the computation efficiency.

[0089] It is known that the calculation time after optimization is only 23% of the original time (150915 / 652938), which is about 77% less. In addition, the number of action instructions was about 10,000 before optimization, which was reduced to about 1,000 after optimization, a reduction of 90%, which significantly reduced the instruction complexity.

[0090] In the description of this specification, the references to terms such as "some possible implementations," "some implementations," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that implementation or example is included in at least one implementation or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same implementation or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more implementations or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different implementations or examples described in this specification, as well as the features of different implementations or examples.

[0091] Regarding the method flowcharts of embodiments of this disclosure, certain operations are described as different steps performed in a certain order. Such flowcharts are illustrative and not restrictive. Some steps described herein may be grouped together and performed in a single operation, some steps may be divided into multiple sub-steps, and some steps may be performed in an order different from that shown herein. The various steps shown in the flowcharts may be implemented in any way by any circuit structure and / or tangible mechanism (e.g., software running on a computer device, hardware (e.g., logic functions implemented by a processor or chip), and / or any combination thereof).

[0092] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0093] Based on the same inventive concept, this disclosure also provides a data processing device corresponding to the data processing method. Since the principle of the device in this disclosure for solving the problem is similar to the data processing method described above in this disclosure, the implementation of the device can refer to the implementation of the method, and repeated details will not be repeated.

[0094] Reference Figure 3 The diagram shown is a schematic representation of a data processing apparatus provided in an embodiment of this disclosure. The apparatus includes: an acquisition module 301, a splitting module 302, and a calculation module 303; wherein,

[0095] The acquisition module 301 is used to acquire the input features and weight parameters of the fully connected layer;

[0096] The splitting module 302 is used to split the input features into multiple input matrices and to split the weight parameters into multiple weight matrices accordingly, wherein at least one of the input matrices has a greater than 1 number of features in the row direction and / or a greater than 1 number of features in the column direction;

[0097] The operation module 303 is used to input multiple input matrices and multiple weight matrices into the PE array for convolution operation to obtain the output features of the fully connected layer.

[0098] Using the aforementioned data processing device, given the input features and weight parameters of the fully connected layer, the input features can be split into multiple input matrices, and the weight parameters can be correspondingly split into multiple weight matrices. After inputting these multiple input matrices and weight matrices into the PE array for convolution calculation, the output features of the fully connected layer can be obtained. During the splitting process, regardless of whether the number of features in the row direction or the number of features in the column direction of the resulting input matrix is ​​greater than 1, a portion of the data input to the fully connected layer in one dimension can be split into rows / columns for subsequent calculations. This allows the PE array to utilize more computing units each time it is started, thereby improving the utilization rate of the PE array.

[0099] In one possible implementation, the row and column distributions of the multiple input matrices are all consistent.

[0100] In one possible implementation, the number of input matrices is greater than or equal to the number of rows in the PE array.

[0101] In one possible implementation, the number of input matrices is an integer multiple of the number of rows in the PE array.

[0102] In one possible implementation, the number of features of the input matrix is ​​less than or equal to the computation threshold of the computational unit of the PE array.

[0103] In one possible implementation, the number of features in the input matrix in the row direction is less than or equal to the computation threshold of the computational units of the PE array in the row direction.

[0104] In one possible implementation, the number of features of the input matrix in the column direction is less than or equal to the computation threshold of the computational unit of the PE array in the column direction.

[0105] In one possible implementation, the input features of the fully connected layer are a one-dimensional vector, and the number of features in the column direction of the weight parameters is the same as the number of features in the one-dimensional vector.

[0106] In one possible implementation, the computation module 303 is configured to input the plurality of input matrices and the plurality of weight matrices into the PE array for convolution operation according to the following steps:

[0107] For each input matrix and its corresponding weight matrix, a calculation unit of the PE array is used to perform a dot product operation on the input matrix and the corresponding weight matrix to obtain the calculation result.

[0108] The results of the operations on each input matrix and its corresponding weight matrix are summarized to obtain the output features of the fully connected layer.

[0109] In one possible implementation, the calculation module 303 is used to summarize the calculation results obtained from each input matrix and its corresponding weight matrix according to the following steps:

[0110] The computational results obtained by the computational units in the same column of the PE array are superimposed to obtain the output features of the fully connected layer.

[0111] It should be noted that the apparatus in this embodiment can implement the various processes of the aforementioned method and achieve the same effects and functions, which will not be elaborated here.

[0112] This disclosure also provides an electronic device, such as... Figure 4 The diagram shown is a schematic representation of an electronic device structure provided in an embodiment of this disclosure, including: a processor 401, a memory 402, and a bus 403. The memory 402 stores machine-readable instructions executable by the processor 401 (e.g., ...). Figure 3 The device acquires the execution instructions corresponding to the acquisition module 301, the splitting module 302, and the calculation module 303, etc. When the electronic device is running, the processor 401 and the memory 402 communicate via the bus 403. When the machine-readable instructions are executed by the processor 401, the following processing is performed:

[0113] Obtain the input features and weight parameters of the fully connected layer;

[0114] The input features are split into multiple input matrices, and the weight parameters are correspondingly split into multiple weight matrices. At least one of the input matrices has a greater than 1 number of features in the row direction and / or a greater than 1 number of features in the column direction.

[0115] Multiple input matrices and multiple weight matrices are input into the PE array for convolution operations to obtain the output features of the fully connected layer.

[0116] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the data processing method described in the above method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0117] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the data processing method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0118] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0119] The various embodiments in this disclosure are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the descriptions of the apparatus, device, and computer-readable storage medium embodiments are simplified because they are substantially similar to the method embodiments; relevant details can be found in the descriptions of the method embodiments.

[0120] The apparatus, device, and computer-readable storage medium provided in this disclosure correspond one-to-one with the method. Therefore, the apparatus, device, and computer-readable storage medium also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the apparatus, device, and computer-readable storage medium will not be repeated here.

[0121] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, apparatus (devices or systems), or computer-readable storage media. Therefore, this disclosure can take the form of a completely hardware implementation, a completely software implementation, or an implementation combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer-readable storage medium implemented on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.

[0122] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices or systems), and computer-readable storage media according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0126] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0127] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. Furthermore, although the operations of the methods of this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0128] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A data processing method, characterized in that, include: Obtain the input features and weight parameters of the fully connected layer, wherein the input features of the fully connected layer are one-dimensional vectors; The input features are split into multiple input matrices, and the weight parameters are correspondingly split into multiple weight matrices. The multiple input matrices are formed by redistributing some features that were originally in the first dimension of the one-dimensional vector to the second and / or third dimensions, such that at least one of the input matrices has more than 1 feature in the row direction and / or more than 1 feature in the column direction. The input matrices and weight matrices are input into the PE array for convolution operation to obtain the output features of the fully connected layer. The input feature is split such that the number of passes required to start the PE array for computation is less than the number of passes required to split the one-dimensional vector only according to the number of rows in the PE array.

2. The method according to claim 1, characterized in that, The row and column distributions of the multiple input matrices are all consistent.

3. The method according to claim 1, characterized in that, The number of input matrices is greater than or equal to the number of rows in the PE array.

4. The method according to claim 1, characterized in that, The number of input matrices is an integer multiple of the number of rows in the PE array.

5. The method according to claim 1, characterized in that, The number of features in the input matrix is ​​less than or equal to the computation threshold of the computational unit of the PE array.

6. The method according to claim 5, characterized in that, The number of features in the input matrix in the row direction is less than or equal to the calculation threshold of the computing unit of the PE array in the row direction.

7. The method according to claim 5, characterized in that, The number of features in the input matrix in the column direction is less than or equal to the computation threshold of the computation unit of the PE array in the column direction.

8. The method according to claim 1, characterized in that, The number of features in the column direction of the weight parameter is the same as the number of features in the one-dimensional vector.

9. The method according to any one of claims 1 to 8, characterized in that, The step of inputting multiple input matrices and multiple weight matrices into a PE array for convolution operation includes: For each input matrix and its corresponding weight matrix, a calculation unit of the PE array is used to perform a dot product operation on the input matrix and the corresponding weight matrix to obtain the calculation result. The results of the operations on each input matrix and its corresponding weight matrix are summarized to obtain the output features of the fully connected layer.

10. The method according to claim 9, characterized in that, The process of summarizing the computational results obtained from each input matrix and its corresponding weight matrix includes: The computational results obtained by the computational units in the same column of the PE array are superimposed to obtain the output features of the fully connected layer.

11. A data processing apparatus, characterized in that, include: The acquisition module is used to acquire the input features and weight parameters of the fully connected layer, wherein the input features of the fully connected layer are one-dimensional vectors. The splitting module is used to split the input features into multiple input matrices and to split the weight parameters into multiple weight matrices accordingly. The multiple input matrices are formed by redistributing some features that were originally in the first dimension of the one-dimensional vector to the second and / or third dimensions, such that at least one of the input matrices has more than 1 feature in the row direction and / or more than 1 feature in the column direction. The computation module is used to input multiple input matrices and multiple weight matrices into the PE array for convolution operations to obtain the output features of the fully connected layer; The splitting of the input features is such that the number of passes required to initiate the operation of the PE array is less than the number of passes required to split the one-dimensional vector only according to the number of rows of the PE array.

12. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is in operation, the processor communicates with the memory via the bus, and the machine-readable instructions, when executed by the processor, perform the data processing method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the data processing method as described in any one of claims 1 to 10.

Images