Processors, methods, chips, and devices
By introducing a replicator and a splitter into the processor, the feature vector is copied and the weight vector is split. Multiple multiply-accumulate arrays are used for parallel processing, which solves the problem of long latency in existing processors and achieves more efficient multiply-accumulate operations.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- TENCENT TECHNOLOGY (SHENZHEN) CO LTD
- Filing Date
- 2024-11-25
- Publication Date
- 2026-05-26
Smart Images
Figure CN122086355A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of chip technology, and in particular to a processor, method, chip, and device. Background Technology
[0002] With the rapid development of artificial intelligence technology and the increasing scale of neural network models, a large number of multiplication and addition operations are involved in the operation of neural network models. Among them, the multiplication and addition operation of two vectors refers to multiplying the elements of two vectors at the same position, and accumulating the two product results to obtain the multiplied result of the two vectors.
[0003] Taking a processor with 4 rows and 4 columns of execution engines as an example, for the feature matrix and weight matrix in the neural network model, the feature matrix includes at least two feature vectors, and the weight matrix includes at least two weight vectors. Both feature vectors and weight vectors have 4 elements. For any feature vector and the nth weight vector, the four feature elements of the feature vector are input into the four execution engines in the nth column, and the four weight elements of the weight vector are input into the four execution engines in the nth column, where n is greater than 0 and not greater than 4. Each execution engine in the nth column multiplies the input feature elements and weight elements, adds the product result to the product result sent by the execution engine in the previous row, and sends it to the execution engine in the next row. Therefore, the fourth execution engine in the nth column adds the four product results to obtain the product of the feature vector and the weight vector. Similarly, we can obtain the multiplication and addition result of each feature vector and each weight vector. Then, we arrange the at least two multiplication and addition results according to the position of the corresponding feature vector in the feature matrix and the position of the corresponding weight vector in the weight matrix. The resulting matrix is the multiplication and addition matrix of the feature matrix and the weight matrix.
[0004] However, the aforementioned processors have long latency, resulting in low processing efficiency. Summary of the Invention
[0005] This application provides a processor, method, chip, and device that reduce latency and improve processing efficiency. The technical solution is as follows:
[0006] On one hand, a processor is provided, the processor including a storage unit, a replicator, a splitter, and m multiply-accumulate arrays, the replicator being connected to the storage unit and each of the m multiply-accumulate arrays respectively, and the splitter being connected to the storage unit and each of the m multiply-accumulate arrays respectively; each multiply-accumulate array includes n multiply-accumulators, where m and n are integers greater than 1;
[0007] The storage unit is used to store feature data and weight data of the neural network model. The feature data includes at least one feature vector, and the weight data includes m×n weight vectors, wherein the number of elements in each feature vector and each weight vector is equal.
[0008] The replicator is used to copy each feature vector to obtain m sets of feature vectors, and input the i-th feature vector in the m sets of feature vectors into each multiply-accumulator in the i-th multiply-accumulator array, where i is a positive integer not greater than m;
[0009] The splitter is used to split the weight data into m groups of weight sub-data, each group of weight sub-data including n weight vectors. The j-th weight vector belonging to the i-th group of weight sub-data is input to the j-th multiply-accumulate in the i-th multiply-accumulate array, where j is a positive integer not greater than n.
[0010] Each multiply-accumulator is used to perform multiply-accumulate operations on the input feature vector and weight vector to obtain the multiply-accumulate result.
[0011] On the other hand, a processing method is provided, wherein the processor includes a storage unit, a replicator, a splitter, and m multiply-accumulate arrays, wherein the replicator is connected to the storage unit and each of the m multiply-accumulate arrays, and the splitter is connected to the storage unit and each of the m multiply-accumulate arrays; each multiply-accumulate array includes n multiply-accumulators, where m and n are integers greater than 1;
[0012] The processing method includes:
[0013] The storage unit stores the feature data and weight data of the neural network model. The feature data includes at least one feature vector, and the weight data includes m×n weight vectors, with each feature vector and each weight vector having the same number of elements.
[0014] For each feature vector, the replicator copies the feature vector to obtain m sets of feature vectors. The i-th feature vector in the m sets of feature vectors is input into each multiply-accumulator in the i-th multiply-accumulator array, where i is a positive integer not greater than m.
[0015] The splitter splits the weight data into m groups of weight sub-data, each group of weight sub-data including n weight vectors. The j-th weight vector belonging to the i-th weight sub-data is input to the j-th multiply-accumulate in the i-th multiply-accumulate array, where j is a positive integer not greater than n.
[0016] Each multiply-accumulator performs a multiply-accumulate operation on the input feature vector and weight vector to obtain the multiply-accumulate result.
[0017] In one possible implementation, the m×n weight vectors in the weight data are arranged sequentially; the splitter splits the weight data into m groups of weight sub-data, each group of weight sub-data including n weight vectors, including:
[0018] The splitter selects n weight vectors from the weight data each time according to the arrangement order of the m×n weight vectors in the weight data, and forms a set of weight sub-data with the selected n weight vectors, until the last n weight vectors are selected from the weight data and the last n weight vectors are formed into the m-th set of weight sub-data.
[0019] In one possible implementation, the number of elements in each feature vector and each weight vector is k, where k is a power of 2; each multiply-accumulate includes k multipliers and log2k groups of adders, the number of adders in each group being half the number of the previous group, and the number of adders in the first group is...
[0020] In one possible implementation, each multiply-accumulate operator performs a multiply-add operation on the input feature vector and weight vector to obtain a multiply-add result, including:
[0021] For each of the multiply-accumulators, the k multipliers perform multiplication operations in parallel on the k feature elements in the feature vector and the weight elements at the same positions in the weight vector, respectively, to obtain k product results.
[0022] The first set of adders is used to accumulate every two product results from the k product results in parallel, to obtain... Each accumulated result is then passed through the next set of adders, and every two accumulated results obtained from the previous set of adders are accumulated in parallel until a single accumulated result is obtained through the last set of adders. The accumulated result obtained by the last set of adders is then determined as the result of the multiplication and addition of the feature vector and the weight vector.
[0023] In one possible implementation, the replicator inputs the i-th feature vector from the m sets of feature vectors to each multiply-accumulate unit in the i-th multiply-accumulate array, including:
[0024] The replicator inputs the i-th feature vector from the m-th feature vector group into the first multiply-accumulator in the i-th multiply-accumulator array;
[0025] Each multiply-accumulator in the i-th multiply-accumulator array inputs the i-th set of feature vectors into the next multiply-accumulator.
[0026] In one possible implementation, a register is connected between every two adjacent multiply-accumulate units in each multiply-accumulate array;
[0027] Each multiply-accumulator in the i-th multiply-accumulator array inputs the i-th set of feature vectors into the next multiply-accumulator, including:
[0028] Each multiply-accumulator in the i-th multiply-accumulator array is used to input the i-th set of feature vectors into the connected register; and through the register, input the i-th set of feature vectors into the next multiply-accumulator.
[0029] In one possible implementation, the processor further includes m delay units, each delay unit including n delay sub-units, each delay sub-unit being connected to the splitter and to a multiply-accumulate unit;
[0030] The splitter inputs the j-th weight vector belonging to the i-th weight sub-data group to the j-th multiply-accumulate unit in the i-th multiply-accumulate array, where j is a positive integer not greater than n, including:
[0031] The splitter inputs the j-th weight vector belonging to the i-th group of weighted sub-data into the j-th delay sub-unit in the i-th delay unit;
[0032] The j-th delay subunit in the i-th delay unit performs a delay operation on the input weight vector and then inputs it to the connected multiply-accumulator.
[0033] In the same delay unit, among the n delay sub-units, the delay duration of each delay sub-unit is equal to the product of the order of each delay sub-unit in its respective delay unit and the unit delay duration.
[0034] In one possible implementation, the replicator copies each feature vector to obtain m sets of feature vectors, including:
[0035] In each clock cycle, the replicator reads a feature vector from the feature data from the storage unit, copies the read feature vector, and obtains the m sets of feature vectors.
[0036] On the other hand, a chip is provided that includes a processor as described in the above aspects.
[0037] On the other hand, a computer device is provided, the computer device including a processor and a memory as described above, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor.
[0038] On the other hand, a computer-readable storage medium is provided that stores at least one computer program, which is loaded and executed by a processor as described above.
[0039] On the other hand, a computer program product is provided, including a computer program that is loaded and executed by a processor as described above.
[0040] In this embodiment, the processor is configured with m multiply-accumulate arrays, each including n multiply-accumulate units, where m and n are integers greater than 1. Furthermore, through the cooperation of storage units, a copyer, and a splitter, the same feature vector is copied and input into each of the m multiply-accumulate arrays. The m×n weight vectors are split into m groups, with each group's weight vector input into a multiply-accumulate array. The input feature vectors and weight vectors are then multiplied and added in parallel using the m multiply-accumulate arrays. This reduces the number of weight vectors that need to be processed in each multiply-accumulate array. Moreover, since the m multiply-accumulate arrays are independent and can be processed in parallel, latency is reduced, and the processing efficiency of the multiply-accumulate operation is improved. Attached Figure Description
[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0042] Figure 1 This is a schematic diagram of the structure of a processor provided in an embodiment of this application;
[0043] Figure 2 This is a schematic diagram of the structure of a multiply-accumulate provided in an embodiment of this application;
[0044] Figure 3 This is a schematic diagram illustrating the delay duration of a multiply-accumulate array provided in an embodiment of this application;
[0045] Figure 4 This is a schematic diagram of a multiplication-addition operation processing flow provided in an embodiment of this application;
[0046] Figure 5 This is a schematic diagram of another processor structure provided in an embodiment of this application;
[0047] Figure 6 This is a schematic diagram of the structure of a delay unit provided in an embodiment of this application;
[0048] Figure 7This is a flowchart of a processing method provided in an embodiment of this application;
[0049] Figure 8 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application;
[0050] Figure 9 This is a schematic diagram of the structure of a server provided in an embodiment of this application. Detailed Implementation
[0051] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.
[0052] It is understood that the terms "first," "second," etc., used in this application may be used to describe various concepts herein, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of this application, a first vector may be referred to as a second vector, and similarly, a second vector may be referred to as a first vector.
[0053] Here, "at least two" means two or more vectors. For example, "at least two vectors" can be any integer number of vectors greater than or equal to two, such as two vectors, three vectors, etc. "Each" refers to each of the at least two vectors. For example, "each vector" refers to each of the at least two vectors. If the at least two vectors are three vectors, then "each vector" refers to each of the three vectors.
[0054] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices) involved in this application have all been fully authorized by the user or relevant parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0055] Figure 1 This is a schematic diagram of a processor provided in an embodiment of this application. See also... Figure 1 The processor includes a storage unit 101, a replicator 102, a splitter 103, and m multiply-accumulate arrays 104. The replicator 102 is connected to each of the storage unit 101 and the m multiply-accumulate arrays 104, and the splitter 103 is connected to each of the storage unit 101 and the m multiply-accumulate arrays 104. Each multiply-accumulate array 104 includes n multiply-accumulate units 1041, where m and n are integers greater than 1.
[0056] The following sections will describe each part of the processor.
[0057] (1) Storage unit 101.
[0058] Storage unit 101 is used to store data to be processed. When data processing is required, data can be read from storage unit 101 and processed. This embodiment of the application is applied to the operation of a neural network model. During the operation of the neural network model, feature data and weight data need to be multiplied and added together to obtain the result. Therefore, storage unit 101 is used to store feature data and weight data. The feature data includes at least one feature vector, and the weight data includes m×n weight vectors, with each feature vector and each weight vector having the same number of elements. For example, each feature vector includes 8 feature elements, and each weight vector includes 8 weight elements. The multiplication and addition operation on the feature data and weight data refers to combining each feature vector in the feature data with each weight vector in the weight data pairwise, and calculating the result of multiplying and adding the feature vector and weight vector in each combination.
[0059] Optionally, the feature data includes m×n feature vectors, that is, the number of feature vectors is equal to the number of weight vectors.
[0060] Optionally, the feature data can be a feature matrix, where each row of the feature matrix represents a feature vector, or each column represents a feature vector. Alternatively, the feature data can also be a list of feature vectors or other forms. The weight data can be a weight matrix, where each row of the weight matrix represents a weight vector, or each column represents a weight vector. Alternatively, the weight data can also be a list of weight vectors or other forms. This application does not limit the format of the feature data and weight data.
[0061] Optionally, the neural network model can be an image classification model, recommendation model, image recognition model, content generation model, or similar type of model. Furthermore, the neural network model includes at least one model layer, which can be a convolutional layer, fully connected layer, etc. Each model layer has feature data and weight data, requiring multiplication and addition operations on the feature data and weight data. The feature data can be feature maps in the neural network model, and the weight features can be the weights of the feature maps. By calculating the multiplication and addition results of the feature data and weight data, a weighted operation can be performed on the feature data.
[0062] Optionally, the storage unit 101 can be volatile memory or non-volatile memory, and can be read-only memory, random access memory, or dynamic random access memory, etc. The type of storage unit 101 is not limited in this application embodiment. In addition, the processor may include one or at least two storage units 101, and the number of storage units 101 is not limited in this application embodiment.
[0063] (2) Copier 102.
[0064] The replicator 102 has the function of copying data, and the replicator 102 is connected to the storage unit 101, so it can read data from the storage unit 101 and then copy it. In addition, the replicator 102 is also connected to each of the m multiply-accumulate arrays 104, so it can input the copied data to any one or at least two multiply-accumulate arrays 104.
[0065] Therefore, in this embodiment, the replicator 102 is used to copy each feature vector in the feature data to obtain m sets of feature vectors. Each copied feature vector is identical to the original feature vector. The m sets of feature vectors are then input to each of the m multiply-accumulate arrays 1041. That is, the i-th feature vector in the m sets of feature vectors is input to each of the i-th multiply-accumulate arrays 1041, where i is a positive integer not greater than m, and therefore the value range of i is [1, m]. In other words, the first set of feature vectors in the m sets of feature vectors is input to each of the first multiply-accumulate arrays 1041, the second set of feature vectors in the m sets of feature vectors is input to each of the second multiply-accumulate arrays 1041, and so on, until the m-th feature vector in the m sets of feature vectors is input to each of the m-th multiply-accumulate arrays 1041.
[0066] It should be noted that to calculate the multiplication and addition result of feature data and weight data, it is necessary to first calculate the multiplication and addition result of each feature and each weight vector. The resulting at least two multiplication and addition results are then arranged according to the position of the corresponding feature vector in the feature data and the position of the corresponding weight vector in the weight data to obtain the final multiplication and addition result of feature data and weight data. Therefore, this embodiment only uses the processing of one feature vector as an example to illustrate the process of multiplying and adding the feature vector with at least two weight vectors. The processing of other feature vectors is similar, and will not be elaborated further in this embodiment.
[0067] Optionally, the replicator 102 reads a feature vector from the feature data from the storage unit 101 in each clock cycle, copies the read feature vector to obtain m identical feature vectors, so as to subsequently calculate the multiplication and summation result of the read feature vector with each weight vector in the weight data. Therefore, the processor calculates the multiplication and summation result of one feature vector with each weight vector in the weight data separately in each clock cycle, and then calculates the multiplication and summation result of the next feature vector with each weight vector in the weight data separately in the next clock cycle, and so on, reading different feature vectors in different clock cycles until all feature vectors in the feature data have been processed, thus completing the multiplication and summation operation of the feature data and weight data. This method can ensure that different feature vectors are processed in different clock cycles, avoiding interference from other feature vectors and improving processing accuracy.
[0068] (3) Splitter 103.
[0069] The splitter 102 has the function of splitting data, and the splitter 103 is connected to the storage unit 101, which can read data from the storage unit 101 and then split it. In addition, the splitter 103 is also connected to each of the m multiply-accumulate arrays 104, and can input the split data to any one or at least two multiply-accumulate arrays 104.
[0070] Therefore, in this embodiment of the application, the splitter 103 is used to split the weight data into m groups of weight sub-data, each group of weight sub-data including n weight vectors, and input the j-th weight vector belonging to the i-th group of weight sub-data to the j-th multiply-accumulate 1041 in the i-th multiply-accumulate array 104, where j is a positive integer not greater than n.
[0071] Optionally, the process of splitting the weight data into m groups of weight sub-data includes: the m×n weight vectors in the weight data are arranged in order; the splitter is used to select n weight vectors from the weight data each time according to the arrangement order of the m×n weight vectors in the weight data, and form a group of weight sub-data with the selected n weight vectors, until the last n weight vectors are selected from the weight data, and the last n weight vectors are formed into the m-th group of weight sub-data.
[0072] For example, if the weight data is a weight matrix, where each row of the weight matrix is a weight vector, and the weight matrix has a total of m×n rows, then the splitter selects n rows of weight vectors from the weight matrix according to the order of the m×n rows of weight vectors to form the first group of weight sub-data. Then, it selects n rows of weight vectors from the remaining (m-1)×n rows to form the second group of weight sub-data, and so on, until the last n rows of weight vectors are selected from the weight matrix to form the m-th group of weight sub-data.
[0073] Alternatively, the weight data can be a weight matrix, where each column is a weight vector. The weight matrix has m×n columns. The splitter selects n columns of weight vectors from the m×n columns of the weight matrix to form the first set of weight sub-data. Then, it selects n columns of weight vectors from the remaining (m-1)×n columns to form the second set of weight sub-data, and so on, until the last n columns of weight vectors are selected from the weight matrix to form the m-th set of weight sub-data.
[0074] The process of inputting the weight vectors into the multiply-accumulator includes: for the first set of weight sub-data, inputting the first weight vector belonging to that weight sub-data into the first multiply-accumulator 1041 in the first multiply-accumulator array 104, and so on, inputting the nth weight vector belonging to that weight sub-data into the nth multiply-accumulator 1041 in the first multiply-accumulator array 104. For the second set of weight sub-data, inputting the first weight vector belonging to that weight sub-data into the first multiply-accumulator 1041 in the second multiply-accumulator array 104, and so on, inputting the nth weight vector belonging to that weight sub-data into the nth multiply-accumulator 1041 in the second multiply-accumulator array 104. Similarly, for the m-th weighted sub-data, the first weight vector belonging to that weighted sub-data is input to the first multiply-accumulate unit 1041 in the m-th multiply-accumulate array 104, and so on, the n-th weight vector belonging to that weighted sub-data is input to the n-th multiply-accumulate unit 1041 in the m-th multiply-accumulate array 104.
[0075] Optionally, the splitter 103 is used to read weight data from the storage unit 101 and split the weight data into m groups of weight sub-data, and input each group of weight sub-data into each multiply-accumulate array 104. Alternatively, it can read from the storage unit 101 in units of weight vectors, and sequentially input the weight vector read each time into a multiply-accumulate array 1041 according to the order of the m multiply-accumulate arrays 104 and each multiply-accumulate unit 1041 in the m multiply-accumulate arrays 104, thereby realizing the input of the j-th weight vector belonging to the i-th weight sub-data group into the j-th multiply-accumulate unit 1041 in the i-th multiply-accumulate array 104. Furthermore, the splitter 103 is used to repeat the above steps in each clock cycle, or only to perform the above steps in the first clock cycle. In subsequent clock cycles, the same weight vector is not read repeatedly, and the multiply-accumulate unit 1041 can perform multiply-accumulate operations based on the weight vector obtained in the first clock cycle.
[0076] (4) Multiply-accumulate array 104.
[0077] The multiply-accumulate array 104 includes n multiply-accumulate units 1041. Each multiply-accumulate unit 1041 performs multiply-accumulate operations on the input feature vector and weight vector to obtain the multiply-accumulate result. For a single multiply-accumulate array 104, the n multiply-accumulate units are independent and can operate in parallel. Therefore, the multiply-accumulate array 104 can take one feature vector and n weight vectors as input and calculate the multiply-accumulate results of the feature vector with the n weight vectors in parallel. Furthermore, after inputting a new feature vector, the multiply-accumulate results of the new feature vector with the n weight vectors can also be calculated in parallel.
[0078] The m multiply-accumulate arrays 104 are also independent and can be operated in parallel. Therefore, after inputting the same m sets of feature vectors into the m multiply-accumulate arrays 104 respectively, the m multiply-accumulate arrays 104 can calculate the multiplication and addition results of this feature vector with different weight vectors in parallel.
[0079] In related technologies, processors used for multiply-accumulate operations typically employ a multiply-accumulate array, which includes multiple rows and columns of execution engines; that is, the multiply-accumulate array is a two-dimensional array. A feature vector and at least two weight vectors are input into this array. At least two feature elements from the feature vector are input to at least two execution engines in the same column, and at least two weight elements from each weight vector are also input to at least two execution engines in the same column. Weight elements from different weight vectors are input to execution engines in different columns. Each execution engine calculates the multiply-accumulate result of a feature element and a weight element, and each execution engine sends the obtained multiply-accumulate result to the execution engine in the next row of the same column for accumulation, until the execution engine in the last row of the same column obtains the multiply-accumulate result of the feature vector and a weight vector. Furthermore, each execution engine also sends the input feature element to the execution engine in the next column of the same row, so that the execution engine in the next column can calculate the multiply-accumulate result of the feature vector and the next weight vector. In the horizontal direction, the execution engine in the previous column needs to send feature elements to the execution engine in the next column of the same row, which results in latency. For example, if the latency of sending feature elements between two execution engines is expressed as unit latency, and assuming the weight data includes x weight vectors, then the horizontal latency is the product of (x-1) and the unit latency. Furthermore, as processor computing power increases, the width and height of the multiply-accumulate array also increase, leading to a further increase in latency.
[0080] In this embodiment, the processor is configured with m multiply-accumulate arrays, each including n multiply-accumulate units, where m and n are integers greater than 1. Furthermore, through the cooperation of storage units, a copyer, and a splitter, the same feature vector is copied and input into each of the m multiply-accumulate arrays. The m×n weight vectors are split into m groups, with each group's weight vector input into a multiply-accumulate array. The input feature vectors and weight vectors are then multiplied and added in parallel using the m multiply-accumulate arrays. This reduces the number of weight vectors that need to be processed in each multiply-accumulate array. Moreover, since the m multiply-accumulate arrays are independent and can be processed in parallel, latency is reduced, and the processing efficiency of the multiply-accumulate operation is improved.
[0081] For example, if the weight data includes m×n weight vectors, and a processor in related technologies is used to perform multiplication and addition operations, the delay time in the horizontal direction is the product of (m×n-1) and the unit delay time. However, if the processor provided in the embodiments of this application is used to perform multiplication and addition operations, the delay time in the horizontal direction is the product of (n-1) and the unit delay time.
[0082] Based on the above embodiments, in one possible implementation, the number of elements in each feature vector and each weight vector is k, where k is a power of 2; each multiply-accumulate 1041 includes k multipliers and log2k groups of adders, the number of adders in each group is half the number of adders in the previous group, and the number of adders in the first group is... Then the number of adders in the last group is 1.
[0083] In one possible implementation, each multiply-accumulate 1041 is used for:
[0084] Using k multipliers, the k feature elements in the feature vector are multiplied in parallel with the weight elements at the same positions in the weight vector, resulting in k product results. Then, using the first set of adders, every two product results are summed in parallel to obtain... Each accumulated result is then passed through the next set of adders, and every two accumulated results obtained from the previous set of adders are accumulated in parallel until a single accumulated result is obtained through the last set of adders. The accumulated result obtained by the last set of adders is then determined as the result of the multiplication and addition of the feature vector and the weight vector.
[0085] The time consumed by the k multipliers performing multiplication operations in parallel is the unit delay time of one multiplier, and the time consumed by each group of adders performing accumulation in parallel is the unit delay time of one adder. Therefore, the total delay time consumed by the multiply-accumulate unit 1041 is the sum of the unit delay time of one multiplier and the unit delay time of log2k adders.
[0086] In one possible implementation, each adder in the first group of adders is connected to two multipliers, for example, the first and second multipliers are connected to the first adder in the first group of adders, the third and fourth multipliers are connected to the second adder in the first group of adders, and so on. Therefore, in the first group of adders, each adder can accumulate two product results from k product results.
[0087] Furthermore, each adder in the q-th group of adders is connected to two adders in the (q-1)-th group, where q is a positive integer not greater than log₂k. For example, each adder in the 2nd group is connected to two adders in the 1st group. That is, the first adder in the 2nd group is connected to the first and second adders in the 1st group, and the second adder in the 2nd group is connected to the third and fourth adders in the 1st group. Therefore, in all groups except the 1st group, each adder can accumulate the two accumulated results obtained from the previous group.
[0088] See Figure 2Taking k=8 as an example, the feature vector fm includes 8 feature elements: fm[0], fm[1], fm[2], fm[3], fm[4], fm[5], fm[6], fm[7], and the weight vector wt includes 8 weight elements: wt[0], wt[1], wt[2], wt[3], wt[4], wt[5], wt[6], wt[7]. The multiply-accumulate 1041 includes 8 multipliers and 3 sets of adders, forming the structure of an accumulation tree. The first set of adders includes 4 adders, the second set of adders includes 2 adders, and the third set of adders includes 1 adder. Following a one-to-one correspondence between feature elements, weight elements, and multipliers, the eight feature elements are input into their respective eight multipliers, and the eight weight elements are input into their respective multipliers. Each multiplier performs a multiplication operation on the feature elements and weight elements, resulting in eight product results. Since the eight multipliers are independent and can process in parallel, the latency of this multiplication process is equal to the unit latency of one multiplier. Next, the four adders in the first group simultaneously accumulate every two product results, resulting in four accumulated results. The latency of this accumulation process is equal to the unit latency of one adder. Finally, the two adders in the second group simultaneously accumulate every two accumulated results, resulting in two accumulated results. The latency of this accumulation process is equal to the unit latency of one adder. Next, the two accumulated results are summed using one adder in the third group to obtain a single accumulated result, which is the multiplication and addition result of the feature vector fm and the weight vector wt. The latency consumed by this accumulation process is equal to the unit latency of one adder. Therefore, the total latency consumed by the multiply-accumulate unit 1041 is the sum of the unit latency of one multiplier and the unit latency of three adders.
[0089] In related technologies, in each execution engine, after multiplication is performed by a multiplier, the product result is added to the accumulation result sent by the previous execution engine by an adder, and then the multiply-add result is sent to the next execution engine to continue the accumulation. Therefore, when the processor has a total of p execution engines, the unit delay time of one multiplier and the unit delay time of (p-1) adders are required in the vertical direction.
[0090] In this embodiment, k multipliers and log2k groups of adders are set in the multiply-accumulator. The k multipliers perform multiplication operations in parallel on the k feature elements in the feature vector and the weight elements at the same position in the weight vector. The adders in the same group accumulate the obtained results in parallel until the multiplication and addition result of the feature vector and weight vector is obtained through the last group of adders. The above parallel processing method can minimize the delay time consumed by the multiplication and addition operation, thereby improving the processing efficiency.
[0091] Furthermore, the processor reduces horizontal latency by processing m multiply-accumulate arrays in parallel, and reduces vertical latency by processing k multipliers and adders in each group in parallel. For feature data and weight data of the same specifications, using the processor provided in this embodiment to perform multiply-accumulate operations significantly reduces the latency from the start of data flowing into the multiply-accumulate array to the last data flowing out, thus improving the processor's computational performance.
[0092] See Figure 3 In the processor, the latency of each multiply-accumulate array 104 is determined by... Figure 3 The solid line 301 in the diagram represents that the latency of each multiply-accumulate array 104 is equal to (m×n-1) times the product of the unit latency between every two multiply-accumulate units 1041, and the sum of the unit latency of one multiplier and the unit latency of log2k adders. Furthermore, since the m multiply-accumulate arrays 104 process in parallel without interfering with each other, the latency of the entire processor is also equal to (m×n-1) times the product of the unit latency between every two multiply-accumulate units 1041, and the sum of the unit latency of one multiplier and the unit latency of log2k adders.
[0093] Based on the above embodiments, in one possible implementation, the replicator 102 is used to input the i-th feature vector from the m copied feature vectors into the first multiply-accumulate unit 1041 in the i-th multiply-accumulate array 104. Each multiply-accumulate unit 1041 in the i-th multiply-accumulate array 104 is used to input the input i-th feature vector into the next multiply-accumulate unit 1041. Here, i is a positive integer, and i is not greater than m.
[0094] That is, in each multiply-accumulate array 104, the first multiply-accumulate unit 1041 not only performs multiply-accumulate operations on the input feature vector and weight vector, but also inputs the input feature vector to the second multiply-accumulate unit 1041. Similarly, the second multiply-accumulate unit 1041 not only performs multiply-accumulate operations on the input feature vector and weight vector, but also inputs the input feature vector to the third multiply-accumulate unit 1041, and so on. This ensures that different multiply-accumulate units 1041 in the same multiply-accumulate array 104 can input the same feature vector and different weight vectors.
[0095] This method of sending feature vectors from one multiply-accumulator 1041 to the next multiply-accumulator 1041 results in processing delay. For example, if the delay between two multiply-accumulators 1041 in sending feature vectors is expressed as a unit delay, then in each multiply-accumulator array 104, the delay in the horizontal direction is the product of (n-1) and the unit delay.
[0096] It should be noted that the above possible implementation methods are illustrated using the i-th group of feature vectors as an example, and the processing of each group of feature vectors is similar, which will not be repeated in the embodiments of this application.
[0097] Based on the above embodiments, the present application provides the following examples:
[0098] Taking m = 2, n = 2, and k = 8 as an example, the processor includes a storage unit 101, a replicator 102, a splitter 103, and two multiply-accumulate arrays 104. The replicator 102 is connected to the storage unit 101 and each of the two multiply-accumulate arrays 104, and the splitter 103 is connected to the storage unit 101 and each of the two multiply-accumulate arrays 104. Each multiply-accumulate array 104 includes two multiply-accumulate units 1041.
[0099] See Figure 4 The feature data is a 4x8 feature matrix:
[0100]
[0101] Each row in the feature matrix is an eigenvector, so the feature matrix consists of 4 eigenvectors, and each eigenvector consists of 8 feature elements.
[0102] The weight data is a weight matrix with 8 rows and 4 columns:
[0103]
[0104] Each column in the weight matrix is a weight vector, so the weight matrix consists of 4 weight vectors, and each weight vector consists of 8 weight elements.
[0105] In the first clock cycle, the replicator 102 reads feature vector 1 from memory cell 101, copies feature vector 1 into two sets of feature vector 1, and inputs the two sets of feature vector 1 into each of the multiply-accumulate units in the two multiply-accumulate arrays. The splitter 103 splits the weight matrix into two sets of weight sub-data. The first set of weight sub-data includes weight vector 1 and weight vector 2, and the second set of weight sub-data includes weight vector 3 and weight vector 4. Weight vector 1 is input into the first multiply-accumulate unit in the first multiply-accumulate array, weight vector 2 is input into the second multiply-accumulate unit in the first multiply-accumulate array, weight vector 3 is input into the first multiply-accumulate unit in the second multiply-accumulate array, and weight vector 4 is input into the second multiply-accumulate unit in the second multiply-accumulate array. Thus, through these four multiply-accumulate units, the multiplication and addition result of feature vector 1 and the four weight vectors is obtained: x 0,0 x 0,1 x 0,2 x 0,3 The principle behind each multiply-accumulate unit's operation of multiplying and adding the feature vector and weight vector can be found in [link to relevant documentation]. Figure 2 This will not be elaborated upon here.
[0106] In the second clock cycle, replicator 102 reads feature vector 2 from storage unit 101 and performs the same operation as for feature vector 1, thereby obtaining the result of multiplying and adding feature vector 2 with the four weight vectors: x 1,0 x 1,1 x 1,2 x 1,3 In the third clock cycle, the replicator 102 reads feature vector 3 from memory cell 101 and performs the same operation as for feature vector 1, thus obtaining the result of multiplying and adding feature vector 3 with the four weight vectors: x 2,0 x 2,1 x 2,2 x 2,3 In the fourth clock cycle, replicator 102 reads feature vector 4 from memory cell 101 and performs the same operation as for feature vector 1, thereby obtaining the result of multiplying and adding feature vector 4 with the four weight vectors: x 3,0 x 3,1 x 3,2 x 3,3 Therefore, by arranging each multiplication-addition result according to the position of the corresponding eigenvector in the feature matrix and the position of the corresponding weight vector in the weight matrix, we can obtain the multiplication-addition matrix of the feature matrix and the weight matrix:
[0107]
[0108] In one possible implementation, see [link to relevant documentation]. Figure 5Each pair of adjacent multiply-accumulators 1041 in each multiply-accumulator array 104 is connected to a register 1042; each multiply-accumulator 1041 in the i-th multiply-accumulator array 104 is used to input the i-th set of feature vectors into the connected register 1042; and through the register 1042, the i-th set of feature vectors is input into the next multiply-accumulator 1041.
[0109] Furthermore, the above possible implementation only uses the i-th multiply-accumulate array 104 as an example, and each multiply-accumulate array 104 is similar. That is, each multiply-accumulate unit 1041 in each multiply-accumulate array 104 is used to input the input feature vector into the connected register 1042, and through register 1042, input the feature vector into the next multiply-accumulate unit 1041. In other words, in each multiply-accumulate array 104, the first multiply-accumulate unit 1041 receives the feature vector sent by the replicator 102 and stores the feature vector into the connected register 1042, i.e., the first register 1042. Then, the second multiply-accumulate unit 1041 reads the feature vector from the first register 1042. After that, the second multiply-accumulate unit 1041 stores the feature vector into the second register 1042, and the third multiply-accumulate unit 1041 reads the feature vector from the second register 1042, and so on.
[0110] In the above process, the delay time consumed from receiving a feature vector from one multiply-accumulate 1041 to receiving a feature vector from the next multiply-accumulate 1041 is a unit delay time. That is, the process of storing and reading the feature vector in each register 1042 will consume a unit delay time, which is equivalent to each register 1042 performing a "one-beat operation". A "one-beat operation" refers to a delay operation that consumes a unit delay time.
[0111] The above Figure 5 In this implementation, register 1042 represents the register used to store the feature vector. In one possible implementation, the feature vector includes at least two feature elements. At least two registers are connected between every two adjacent multiply-accumulate units 1041 in each multiply-accumulate array 104, and the number of registers is equal to the number of feature elements in the feature vector. The replicator 102 inputs at least two feature elements from the feature vector to the first multiply-accumulate unit 1041 in the multiply-accumulate array 104. Each multiply-accumulate unit 1041 inputs at least two feature elements to the at least two connected registers, and the next multiply-accumulate unit 1041 reads at least two feature elements from the at least two registers. This process is equivalent to each register performing a clock cycle, and the time consumed by this process is the unit delay time.
[0112] In one possible implementation, the processor further includes m delay units 105, each delay unit 105 comprising n delay sub-units 1051. Each delay sub-unit 1051 is connected to a splitter 103 and to a multiply-accumulate unit 1041. Therefore, the m delay units 105 correspond one-to-one with the m multiply-accumulate arrays 104, and the n delay sub-units 1051 in each delay unit 105 are respectively connected to the n multiply-accumulate units 1041 in the corresponding multiply-accumulate array 104. Furthermore, among the n delay sub-units 1051 in the same delay unit 105, the delay duration of each delay sub-unit 1051 is equal to the product of its order within its respective delay unit 105 and the unit delay duration. That is, in the same delay unit 105, the delay duration of the first delay subunit 1051 is equal to the unit delay duration, the delay duration of the second delay subunit 1051 is equal to twice the unit delay duration, and so on, the delay duration of the nth delay subunit 1051 is equal to n times the unit delay duration.
[0113] The splitter 103 is used to input the n weight vectors belonging to the i-th weight sub-data group to the n delay sub-units 1051 in the i-th delay unit 105 respectively; each delay sub-unit 1051 in the i-th delay unit 105 is used to perform a delay operation on the input weight vector and then input it to the connected multiply-accumulate unit 1041.
[0114] That is, the splitter 103 is used to input the j-th weight vector belonging to the i-th weight sub-data group to the j-th delay sub-unit 1051 in the i-th delay unit 105; the j-th delay sub-unit 1051 in the i-th delay unit 105 is used to perform a delay operation on the input weight vector and then input it to the connected multiply-accumulate unit 1041.
[0115] In this embodiment, the starting point is the time when the copier 103 copies the i-th group of feature vectors, which is also the time when the splitter 103 splits the n weight vectors in each group. The time consumed by inputting the feature vector or weight vector into the first multiply-accumulate unit 1041 of the i-th multiply-accumulate array 104 can also be considered as the unit delay time. Therefore, starting from this starting point, after a unit delay time, the feature vector can be input into the first multiply-accumulate unit 1041 of the i-th multiply-accumulate array 104, and the first weight vector can be input into the i-th multiply-accumulate array... The first multiply-accumulator 1041 in the 104 array, after a delay of twice the unit time, the feature vector can be input to the second multiply-accumulator 1041 in the 1i-th multiply-accumulator array 104, and the second weight vector can be input to the second multiply-accumulator 1041 in the 1i-th multiply-accumulator array 104, and so on. After a delay of n times the unit time, the feature vector can be input to the nth multiply-accumulator 1041 in the 1i-th multiply-accumulator array 104, and the nth weight vector can be input to the nth multiply-accumulator 1041 in the 1i-th multiply-accumulator array 104. Therefore, by combining the slapping operation of register 1042 and the delay operation of delay subunit 1051, it can be ensured that the feature vector and the corresponding weight vector can be simultaneously input into the same multiply-accumulate unit 1041, and the multiply-accumulate unit 1041 performs the multiply-add operation, which ensures the synchronization of the feature vector and the weight vector, and ensures that the multiply-accumulate unit 1041 can perform the multiply-add operation on the feature vector and the weight vector as early as possible, without causing processing delay due to the late input of either vector.
[0116] In one possible implementation, each of the n delay subunits 1051 in the same delay unit 105 includes at least one register, and a pausing operation is performed through at least one register to achieve the purpose of delay.
[0117] Furthermore, the number of registers in each delay subunit 1051 is equal to the order of each delay subunit 1051 within its respective delay unit 105. See also Figure 6 In the same delay unit 105, the first delay subunit 1051 includes one register, the second delay subunit 1051 includes two registers, and so on, with the nth delay subunit 1051 including n registers.
[0118] The splitter 103 inputs the weight vector into the delay sub-unit 1051. After passing through each register in the delay sub-unit 1051, the weight vector is input into the multiply-accumulate unit 1041. Therefore, the delay duration of the weight vector is equal to the product of the number of registers and the unit delay duration.
[0119] Based on the above embodiments, Figure 7This is a flowchart of a processing method provided in an embodiment of this application. The processing method is executed by the processor shown in the above embodiment. The processor includes a storage unit, a replicator, a splitter, and m multiply-accumulate arrays. The replicator is connected to the storage unit and each of the m multiply-accumulate arrays, and the splitter is connected to the storage unit and each of the m multiply-accumulate arrays. Each multiply-accumulate array includes n multiply-accumulators, where m and n are integers greater than 1. See also... Figure 7 The method includes:
[0120] 701. The storage unit stores the feature data and weight data of the neural network model. The feature data includes at least one feature vector, and the weight data includes m×n weight vectors, and the number of elements in each feature vector and each weight vector is equal.
[0121] 702. The replicator copies each feature vector to obtain m sets of feature vectors. The i-th feature vector in the m sets of feature vectors is input into each multiply-accumulator in the i-th multiply-accumulator array, where i is a positive integer not greater than m.
[0122] 703. The splitter splits the weight data into m groups of weight sub-data. Each group of weight sub-data includes n weight vectors. The j-th weight vector belonging to the i-th group of weight sub-data is input into the j-th multiply-accumulate in the i-th multiply-accumulate array, where j is a positive integer not greater than n.
[0123] 704. Each multiply-accumulater performs a multiply-accumulate operation on the input feature vector and weight vector to obtain the multiply-accumulate result.
[0124] In one possible implementation, the m×n weight vectors in the weight data are arranged sequentially; the splitter splits the weight data into m groups of weight sub-data, each group of weight sub-data including n weight vectors, including:
[0125] The splitter selects n weight vectors from the weight data each time according to the order of the m×n weight vectors in the weight data. The selected n weight vectors form a set of weight sub-data, until the last n weight vectors are selected from the weight data and the last n weight vectors form the m-th set of weight sub-data.
[0126] In one possible implementation, the number of elements in each feature vector and each weight vector is k, where k is a power of 2; each multiply-accumulate unit includes k multipliers and log2k groups of adders, where the number of adders in each group is half the number of adders in the previous group, and the number of adders in the first group is...
[0127] In one possible implementation, each multiply-accumulate operator performs a multiply-accumulate operation on the input feature vector and weight vector to obtain the multiply-accumulate result, including:
[0128] For each multiply-accumulator, k multipliers are used to perform multiplication operations on the k feature elements in the feature vector with the weight elements in the weight vector at the same position in parallel, resulting in k product results.
[0129] The first set of adders performs parallel summation of every two product results from the k product results to obtain... Each accumulated result is then passed through the next set of adders, and every two accumulated results obtained from the previous set of adders are accumulated in parallel until a single accumulated result is obtained through the last set of adders. The accumulated result obtained by the last set of adders is then determined as the result of the multiplication and addition of the feature vector and the weight vector.
[0130] In one possible implementation, the replicator inputs the i-th set of feature vectors from the m sets of feature vectors into each multiply-accumulate unit in the i-th multiply-accumulate array, including:
[0131] The replicator inputs the i-th feature vector from the m feature vectors into the first multiply-accumulator in the i-th multiply-accumulator array;
[0132] Each multiply-accumulator in the i-th multiply-accumulator array inputs the i-th set of feature vectors into the next multiply-accumulator.
[0133] In one possible implementation, a register is connected between every two adjacent multiply-accumulate units in each multiply-accumulate array;
[0134] A register is connected between every two adjacent multiply-accumulate units in each multiply-accumulate array;
[0135] Each multiply-accumulator in the i-th multiply-accumulator array inputs the i-th set of feature vectors into the next multiply-accumulator, including:
[0136] Each multiply-accumulator in the i-th multiply-accumulator array is used to input the i-th set of feature vectors into the connected register; and through the register, input the i-th set of feature vectors into the next multiply-accumulator.
[0137] In one possible implementation, the processor further includes m delay units, each delay unit including n delay sub-units, each delay sub-unit being connected to a splitter and a multiply-accumulate unit;
[0138] The splitter inputs the j-th weight vector belonging to the i-th weight sub-data group to the j-th multiply-accumulate unit in the i-th multiply-accumulate array, where j is a positive integer not greater than n, including:
[0139] The splitter inputs the j-th weight vector belonging to the i-th weight sub-data group into the j-th delay sub-unit in the i-th delay unit;
[0140] The j-th delay sub-unit in the i-th delay unit performs a delay operation on the input weight vector and then inputs it to the connected multiply-accumulator.
[0141] In the same delay unit, among the n delay sub-units, the delay duration of each delay sub-unit is equal to the product of the order of each delay sub-unit in its respective delay unit and the unit delay duration.
[0142] In one possible implementation, the copier copies each feature vector, resulting in m sets of feature vectors, including:
[0143] In each clock cycle, the replicator reads a feature vector from the feature data in the storage unit, copies the read feature vector, and obtains m sets of feature vectors.
[0144] In this embodiment, the processor is configured with m multiply-accumulate arrays, each including n multiply-accumulate units, where m and n are integers greater than 1. Furthermore, through the cooperation of storage units, a copyer, and a splitter, the same feature vector is copied and input into each of the m multiply-accumulate arrays. The m×n weight vectors are split into m groups, with each group's weight vector input into a multiply-accumulate array. The input feature vectors and weight vectors are then multiplied and added in parallel using the m multiply-accumulate arrays. This reduces the number of weight vectors that need to be processed in each multiply-accumulate array. Moreover, since the m multiply-accumulate arrays are independent and can be processed in parallel, latency is reduced, and the processing efficiency of the multiply-accumulate operation is improved.
[0145] Based on the above embodiments, this application also provides a chip. The chip includes the processor shown in the above embodiments.
[0146] This application also provides a computer device, which includes a processor and a memory. The memory stores at least one computer program, which is loaded and executed by the processor shown in the above embodiments.
[0147] Figure 8 A schematic diagram of the structure of a terminal 800 provided in an exemplary embodiment of this application is shown. The terminal can be a computer device as described in the above embodiments.
[0148] The terminal 800 includes a processor 801 and a memory 802. The processor 801 is as shown in the above embodiment.
[0149] Processor 801 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 801 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 801 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 801 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 801 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.
[0150] The memory 802 may include one or more computer-readable storage media, which may be non-transitory. The memory 802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 802 are used to store at least one computer program, which is used by the processor 801 to implement the processing method provided in the method embodiments of this application.
[0151] In some embodiments, the terminal 800 may also optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, memory 802, and peripheral device interface 803 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 803 via a bus, signal line, or circuit board. Optionally, the peripheral device includes at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, and a power supply 807.
[0152] Peripheral interface 803 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 801 and memory 802.
[0153] The radio frequency (RF) circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 804 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals.
[0154] Display screen 805 is used to display a UI (User Interface). This UI may include graphics, text, icons, video, and any combination thereof. When display screen 805 is a touch display screen, it also has the ability to acquire touch signals on or above the surface of display screen 805.
[0155] The camera assembly 806 is used to capture images or videos. Optionally, the camera assembly 806 includes a front-facing camera and a rear-facing camera. The front-facing camera is located on the front panel of the terminal 800, and the rear-facing camera is located on the back of the terminal 800.
[0156] Power supply 807 is used to supply power to the various components in terminal 800. Power supply 807 can be AC power, DC power, a disposable battery, or a rechargeable battery.
[0157] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on terminal 800 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.
[0158] Figure 9 This is a schematic diagram of a server structure provided in an embodiment of this application. The server can be the computer device described in the above embodiments. The server 900 can vary significantly due to different configurations or performance, and may include one or more Central Processing Units (CPUs) 901 and one or more memories 902. The memories 902 store at least one computer program, which is loaded and executed by the processor 901, as shown in the above embodiments. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.
[0159] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by the processor shown in the above embodiments.
[0160] This application also provides a computer program product, including a computer program that is loaded and executed by the processor shown in the above embodiments.
[0161] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0162] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present application should be included within the protection scope of the present application.
Claims
1. A processor, characterized in that, The processor includes a storage unit, a replicator, a splitter, and m multiply-accumulate arrays. The replicator is connected to the storage unit and each of the m multiply-accumulate arrays, and the splitter is connected to the storage unit and each of the m multiply-accumulate arrays. Each multiply-accumulate array includes n multiply-accumulators, where m and n are integers greater than 1. The storage unit is used to store feature data and weight data of the neural network model. The feature data includes at least one feature vector, and the weight data includes m×n weight vectors, wherein the number of elements in each feature vector and each weight vector is equal. The replicator is used to copy each feature vector to obtain m sets of feature vectors, and input the i-th feature vector in the m sets of feature vectors into each multiply-accumulator in the i-th multiply-accumulator array, where i is a positive integer not greater than m; The splitter is used to split the weight data into m groups of weight sub-data, each group of weight sub-data including n weight vectors. The j-th weight vector belonging to the i-th group of weight sub-data is input to the j-th multiply-accumulate in the i-th multiply-accumulate array, where j is a positive integer not greater than n. Each multiply-accumulator is used to perform multiply-accumulate operations on the input feature vector and weight vector to obtain the multiply-accumulate result.
2. The processor according to claim 1, characterized in that, The m×n weight vectors in the weight data are arranged in order; The splitter is configured to select n weight vectors from the weight data each time according to the arrangement order of the m×n weight vectors in the weight data, and form a set of weight sub-data from the selected n weight vectors, until the last n weight vectors are selected from the weight data and the last n weight vectors are formed into the m-th set of weight sub-data.
3. The processor according to claim 1, characterized in that, The number of elements in each feature vector and each weight vector is k, where k is a power of 2; each multiply-accumulate unit includes k multipliers and log2k groups of adders, the number of adders in each group being half the number of adders in the previous group, and the number of adders in the first group is...
4. The processor according to claim 3, characterized in that, Each multiply-accumulator is used for: The k multipliers are used to perform multiplication operations in parallel on the k feature elements in the feature vector and the weight elements in the weight vector at the same position, resulting in k product results. The first set of adders is used to accumulate every two product results from the k product results in parallel, to obtain... Each set of adders accumulates two results in parallel, until a single result is obtained from the last set of adders. The result obtained from the last set of adders is then determined as the product of the feature vector and the weight vector.
5. The processor according to claim 1, characterized in that, The replicator is used to input the i-th feature vector from the m-th feature vector group into the first multiply-accumulator in the i-th multiply-accumulator array; Each multiply-accumulator in the i-th multiply-accumulator array is used to input the i-th set of feature vectors into the next multiply-accumulator.
6. The processor according to claim 5, characterized in that, A register is connected between every two adjacent multiply-accumulate units in each multiply-accumulate array; Each multiply-accumulator in the i-th multiply-accumulator array is used to input the i-th set of feature vectors into the connected register; and through the register, input the i-th set of feature vectors into the next multiply-accumulator.
7. The processor according to claim 5, characterized in that, The processor further includes m delay units, each delay unit includes n delay sub-units, each delay sub-unit is connected to the splitter and to a multiply-accumulate unit; The splitter is used to input the j-th weight vector belonging to the i-th group of weight sub-data into the j-th delay sub-unit in the i-th delay unit; The j-th delay subunit in the i-th delay unit is used to perform a delay operation on the input weight vector and then input it to the connected multiply-accumulator. In the same delay unit, among the n delay sub-units, the delay duration of each delay sub-unit is equal to the product of the order of each delay sub-unit in its respective delay unit and the unit delay duration.
8. The processor according to claim 1, characterized in that, The replicator is configured to read a feature vector from the feature data from the storage unit in each clock cycle, and copy the read feature vector to obtain the m sets of feature vectors.
9. A processing method, characterized in that, The processor includes a storage unit, a replicator, a splitter, and m multiply-accumulate arrays. The replicator is connected to the storage unit and each of the m multiply-accumulate arrays, and the splitter is connected to the storage unit and each of the m multiply-accumulate arrays. Each multiply-accumulate array includes n multiply-accumulators, where m and n are integers greater than 1. The processing method includes: The storage unit stores the feature data and weight data of the neural network model. The feature data includes at least one feature vector, and the weight data includes m×n weight vectors, with each feature vector and each weight vector having the same number of elements. For each feature vector, the replicator copies the feature vector to obtain m sets of feature vectors. The i-th feature vector in the m sets of feature vectors is input into each multiply-accumulator in the i-th multiply-accumulator array, where i is a positive integer not greater than m. The splitter splits the weight data into m groups of weight sub-data, each group of weight sub-data including n weight vectors. The j-th weight vector belonging to the i-th weight sub-data is input to the j-th multiply-accumulate in the i-th multiply-accumulate array, where j is a positive integer not greater than n. Each multiply-accumulator performs a multiply-accumulate operation on the input feature vector and weight vector to obtain the multiply-accumulate result.
10. The method according to claim 9, characterized in that, The m×n weight vectors in the weight data are arranged in order; the splitter splits the weight data into m groups of weight sub-data, each group of weight sub-data including n weight vectors, including: The splitter selects n weight vectors from the weight data each time according to the arrangement order of the m×n weight vectors in the weight data, and forms a set of weight sub-data with the selected n weight vectors, until the last n weight vectors are selected from the weight data and the last n weight vectors are formed into the m-th set of weight sub-data.
11. The method according to claim 9, characterized in that, The number of elements in each feature vector and each weight vector is k, where k is a power of 2; each multiply-accumulate unit includes k multipliers and log2k groups of adders, the number of adders in each group being half the number of adders in the previous group, and the number of adders in the first group is...
12. The method according to claim 11, characterized in that, Each multiply-accumulate unit performs a multiply-accumulate operation on the input feature vector and weight vector to obtain the multiply-accumulate result, including: For each of the multiply-accumulators, the k multipliers perform multiplication operations in parallel on the k feature elements in the feature vector and the weight elements at the same positions in the weight vector, respectively, to obtain k product results. The first set of adders is used to accumulate every two product results from the k product results in parallel, to obtain... Each accumulated result is then passed through the next set of adders, and every two accumulated results obtained from the previous set of adders are accumulated in parallel until a single accumulated result is obtained through the last set of adders. The accumulated result obtained by the last set of adders is then determined as the result of the multiplication and addition of the feature vector and the weight vector.
13. A chip, characterized in that, The chip includes the processor as described in any one of claims 1 to 8.
14. A computer device, characterized in that, The computer device includes a processor and a memory as described in any one of claims 1 to 8, wherein the memory stores at least one computer program, the at least one computer program being loaded and executed by the processor.
15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor as described in any one of claims 1 to 8.
16. A computer program product, comprising a computer program, characterized in that, The computer program is loaded and executed by the processor as described in any one of claims 1 to 8.