Chip and electronic device

Abstract

The application provides a chip and an electronic device, and relates to the chip field, and the chip comprises: a plurality of processing units, which are used for parallel processing of input data input into each processing unit; and any processing unit, which comprises: a first processing module, which is used for performing a first target operation matched with data precision on the input data based on the data precision to which the input data belongs, so as to obtain first target output; wherein the first target operation comprises at least one of a multiplication operation, an accumulation operation and a quantization operation; and a second processing module, which is used for performing a second target operation matched with the data precision on the first target output, so as to obtain second target output; wherein the second target operation comprises a column compression operation and / or a row compression operation. In this way, through parallel work of the plurality of processing units, multiple copies of the input data can be simultaneously distributed to different processing units for independent processing, so that the waiting time in the cascade architecture is effectively eliminated, the overall processing delay is significantly reduced, and the throughput is improved.

Description

Technical Field

[0001] This application relates to the field of chip technology, and more particularly to a chip and an electronic device. Background Technology

[0002] The multiply-accumulate (MAC) unit array in artificial intelligence (AI) chips is a core module for deep learning computing, primarily responsible for performing matrix multiplication and accumulation operations, which dominate neural network models. With the rapid growth in AI computing power demands, current development of MAC unit arrays is focusing on key technologies such as high-density integration, low-power design, reconfigurable architecture, and in-memory computing, aiming to achieve a balance between higher energy efficiency and stronger computing power in a wide range of application scenarios, from edge intelligent devices to hyperscale data centers. Summary of the Invention

[0003] This application proposes a chip and an electronic device to at least partially solve one of the technical problems in the related art.

[0004] One embodiment of this application proposes a chip, including: a plurality of processing units for parallel processing of input data input to each of the processing units; wherein, any one of the processing units includes: a first processing module and a second processing module; The first processing module is configured to perform a first target operation on the input data based on the data precision to which the input data belongs, and obtain a first target output; wherein the first target operation includes at least one of multiplication operation, accumulation operation, and quantization operation; The second processing module is used to perform a second target operation on the first target output that matches the data precision, to obtain a second target output; wherein the second target operation includes column compression operation and / or row compression operation.

[0005] Another embodiment of this application proposes an electronic device including the chip described in the foregoing aspect.

[0006] The chip and electronic device proposed in this application, through the parallel operation of multiple processing units, can simultaneously distribute multiple sets of input data to different processing units for independent processing, effectively eliminating the waiting time in the cascaded architecture, significantly reducing the overall processing latency, and improving the system throughput efficiency. Specifically, the first and second processing modules within each processing unit can perform targeted processing on the input data according to its precision type, enabling the chip to possess multi-precision computing capabilities, flexibly adapting to the needs of different application scenarios, thereby enhancing the adaptability and flexibility of data processing.

[0007] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0008] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A schematic diagram of a chip structure provided for an exemplary embodiment of this application; Figure 2 A schematic diagram of another chip structure provided for an exemplary embodiment of this application; Figure 3 A schematic diagram of a matrix multiplication calculation process provided for an exemplary embodiment of this application; Figure 4 A schematic diagram of the structure of a first processing submodule provided for an exemplary embodiment of this application; Figure 5 A schematic diagram of the structure of a second processing submodule provided for an exemplary embodiment of this application; Figure 6 A schematic diagram of the structure of another chip provided for an exemplary embodiment of this application; Figure 7 A schematic diagram of the structure of a second processing module provided for an exemplary embodiment of this application; Figure 8 A schematic diagram of another chip structure provided for an exemplary embodiment of this application; Figure 9 A schematic diagram of the ACCREG reading process provided for an exemplary embodiment of this application; Figure 10 This is a schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. Detailed Implementation

[0009] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0010] Among related technologies, MAC unit array technology mainly includes the following two types: The first type is the Systolic Array: The Systolic Array arranges MAC units in a regular two-dimensional grid, each unit being called a Processing Element (PE). For data transmission, precise data flow control is achieved between PEs through pipelines and synchronized clocks. Data (such as weight matrices and input matrices) is input from the edges of the MAC unit array (e.g., the top and left) and passed step-by-step along fixed paths, such as flowing right and down. This ordered data flow ensures computational stability.

[0011] Among them, systolic arrays can efficiently support tensor operations in deep learning, providing powerful computing support for the training and inference of large-scale neural network models. Taking the field of intelligent driving as an example, using large-scale systolic arrays can achieve more than 1000 TOPS (Tera Operations Per Second), ensuring the efficient operation of real-time image processing in intelligent driving systems and ensuring that vehicles can quickly and accurately perceive their surroundings and make decisions.

[0012] The second type is the broadcast-based architecture: This architecture distributes data (such as weight matrices and input matrices) simultaneously to all MAC units via a broadcast bus. Each MAC unit can perform calculations independently, and the results are aggregated through a local bus or a tree structure. This architecture fully leverages the advantages of parallel computing, improving computational efficiency.

[0013] In typical applications, the Graphics Processing Unit (GPU) utilizes a Single Instruction Multiple Threads (SIMT) architecture to achieve general-purpose parallel computing through shared memory and register broadcasting. This architecture efficiently supports multi-dimensional matrix operations and is widely used in fields such as graphics rendering and scientific computing. The Central Processing Unit (CPU) employs a Single Instruction Multiple Data (SIMD) unit, which uses broadcast registers to perform vector operations, improving CPU performance in tasks such as multimedia and numerical computation.

[0014] However, the above solution has at least the following problems: 1. Computational efficiency needs to be improved. For example, the PEs in the pulsating array are processed in a pipelined series, and the data is passed level by level, resulting in high overall processing latency and limited throughput efficiency. 2. Mixed-precision computation suffers from poor flexibility, exhibiting significant limitations in mixed-precision matrix computation. The precision modes of the basic computational units in related technologies are relatively fixed (e.g., A8W4, where A8 refers to an 8-bit width for each element in the AI ​​model's input matrix, and W4 refers to a 4-bit width for each element in the weight matrix). This makes it difficult to flexibly adjust to the diverse precision requirements of different deep learning models. For example, some models do not require high precision in the early stages of training and can use low-precision computation to speed up the process; however, in the later stages of training or inference, high-precision computation is needed to ensure accuracy. The related technologies struggle to dynamically switch precision modes according to these different stage requirements, limiting their application in the training and inference of complex models.

[0015] 3. High power consumption.

[0016] Therefore, in view of at least one of the problems existing in the above-mentioned related technologies, this application proposes a chip and an electronic device.

[0017] The following describes the chip and electronic device according to embodiments of this application with reference to the accompanying drawings.

[0018] Figure 1 This is a schematic diagram of the structure of a chip provided for an exemplary embodiment of this application.

[0019] The chips provided in this application include AI chips, which include, but are not limited to: Graphics Processing Unit (GPU), Tensor Processing Unit (TPU), Neural Processing Unit (NPU), Data Processing Unit (DPU), Field-Programmable Gate Array (FPGA), Application-Specific Integrated Circuit (ASIC), In-Memory Computing / Compute-in-Memory, Neuromorphic Chip, etc., which will not be listed here.

[0020] like Figure 1As shown, the chip may include: multiple processing elements (PEs) 100 for parallel processing of input data input to each processing element 100; wherein each processing element 100 includes: a first processing module 110 and a second processing module 120. The first processing module 110 is used to perform a first target operation matching the data precision of the input data based on the data precision of the input data, and obtain a first target output; wherein the first target operation includes at least one of multiplication operation, accumulation operation, and quantization operation.

[0021] The input data refers to the data currently input to the processing unit 100, and the input data is different for different processing units 100.

[0022] The input data of the multiple processing units 100 is obtained by splitting the total input data of the chip. The input data of different processing units 100 do not overlap, so that the input data input to themselves can be processed in parallel by multiple processing units 100.

[0023] As an example, taking the chip's total input data as including the input matrix A determined by the input features based on the deep learning model and the weight matrix W of the deep learning model, the chip performs tensor computation (A×W) of A and W. A and W can be split into multiple parts and input to multiple processing units (PEs) in parallel.

[0024] The input features include, but are not limited to, text features, image features, audio features, and video features. The input features are related to the application scenario of the deep learning model. For example, if the deep learning model is used for text classification, the input features can be text features; if it's used for image classification, the input features can be image features; and if it's used for speech recognition, the input features can be audio features.

[0025] For example, taking a tensor with 32 PEs, an 8×32 matrix A, and a 32×32 matrix W, A and W can be split into 32 parts. The input data of each PE includes a submatrix in A (such as a 2×32 matrix) and a submatrix in W (such as a 32×4 matrix). The output data of each PE is a 2×4 matrix. The output data of the 32 PEs are concatenated to obtain an 8×32 matrix. Thus, multiple PEs can be used to process the split input data in parallel to improve the efficiency of tensor computation.

[0026] The data precision (or data type, data format) of the input data includes, but is not limited to: A8W4 (each element in A has a bit width of 8 bits, and each element in W has a bit width of 4 bits), A8W8 (each element in A has a bit width of 8 bits, and each element in W has a bit width of 8 bits), A8W16 (each element in A has a bit width of 8 bits, and each element in W has a bit width of 16 bits), A16W4 (each element in A has a bit width of 16 bits, and each element in W has a bit width of 4 bits), A16W8 (each element in A has a bit width of 16 bits, and each element in W has a bit width of 8 bits), A16W16 (each element in A has a bit width of 16 bits, and each element in W has a bit width of 16 bits), etc.

[0027] In this embodiment of the application, the first processing module 110 is used to perform a first target operation on the input data that matches the data precision of the input data of the processing unit 100 to obtain a first target output; wherein, the first target operation includes at least one of multiplication operation, accumulation operation and quantization operation.

[0028] The second processing module 120 is used to perform a second target operation on the first target output that matches the data precision, to obtain the second target output; wherein the second target operation includes column compression operation and / or row compression operation.

[0029] In this embodiment of the application, the second processing module 120 is used to perform a second target operation on the first target output of the first processing module 110 that matches the above-mentioned data precision, so as to obtain a second target output.

[0030] In any embodiment of this application, the second processing module 120 is configured to perform a second target operation matching the data precision of the first target output of the first processing module 110 when the data precision of the input data of its processing unit 100 does not match the reference precision, thereby obtaining a second target output. The second target operation adapted to different data precisions is different.

[0031] The reference accuracy is a pre-set data accuracy; for example, the reference accuracy is A8W4.

[0032] In any embodiment of this application, the second processing module 120 is further configured to directly use the first target output of the first processing module 110 as the second target output when the data precision of the input data of its processing unit 100 matches the reference precision, that is, without performing a compression operation on the first target output.

[0033] This allows for targeted processing of input data based on different data precision levels, thereby improving the chip's processing accuracy.

[0034] The chip in this embodiment of the application, through the parallel operation of multiple processing units, can simultaneously distribute multiple sets of input data to different processing units for independent processing, effectively eliminating the waiting time in the cascaded architecture, significantly reducing the overall processing latency, and improving the system throughput efficiency. Specifically, the first and second processing modules within each processing unit can perform targeted processing on the input data according to its precision type, enabling the chip to possess multi-precision computing capabilities, flexibly adapting to the needs of different application scenarios, thereby enhancing the adaptability and flexibility of data processing.

[0035] As one possible implementation method, Figure 2 This is a schematic diagram of the structure of another chip provided for an exemplary embodiment of this application.

[0036] It should be noted that the steps executed by each processing unit in the chip can be executed individually, or they can be executed together in combination with any embodiment of this application or any possible implementation in the embodiment, or they can be executed together in combination with any technical solution in related technologies. The embodiments of this application do not limit this.

[0037] like Figure 2 As shown, the chip may include multiple processing units 100 for parallel processing of input data input to each processing unit 100; wherein each processing unit 100 includes: a first processing module 110 and a second processing module 120. The first processing module 110 includes multiple first processing sub-modules 111 that perform parallel processing, and a second processing sub-module 112; wherein the first processing sub-module 111 is used to perform a multiplication operation on the input data with a precision matching the data to obtain a first intermediate output.

[0038] As one possible implementation, taking the chip's total input data as including an input matrix A determined by input features based on a deep learning model and a weight matrix W of the deep learning model, and the chip performing tensor computation (A×W) of A and W as an example, A and W can be split into multiple parts of input data and input to multiple processing units (PEs) in parallel. For example, taking a number of PEs of 32, A as an 8×32 matrix, and W as a 32×32 matrix, A and W can be split into 32 parts of input data. Each PE's input data includes a 2×32 submatrix of A and a 32×4 submatrix of W. Each first processing submodule 111 is used to perform a multiplication operation on a portion of the input data, matching the precision of the input data; wherein the input elements of different first processing submodules 111 are different.

[0039] As an example, taking a number of 32 first processing submodules 111 as an example, the input data (a 2×32 submatrix in A and a 32×4 submatrix in W) can be further divided into 32 parts, which are then input into 32 first processing submodules 111 for parallel computation. The small matrix input to each first submodule 111 can be a 2×1 submatrix in A and a 1×4 submatrix in W.

[0040] For example, taking the deployment of two vectors, 2×1 and 1×4, in each first processing submodule 111 to perform an outer product as an example, the calculation process for matrix multiplication of 2×4 and 4×4 matrices can be as follows: Figure 3 The four first processing submodules shown are implemented, wherein the first processing submodules are in Figure 3 The matrix is ​​labeled TILE. The result of each TILE is a 2×4 partial sum array. By summing the results of the four parallel TILEs, the matrix multiplication of the two vectors, 2×4 and 4×4, can be completed.

[0041] In any embodiment of this application, the input data of each processing unit 100 may include multiple first sub-matrices in the input matrix A determined based on the input features of the deep learning model and multiple second sub-matrices in the weight matrix W of the deep learning model. The number of first sub-matrices is the same as the number of second sub-matrices, and the number of first sub-matrices is also the same as the number of first processing sub-modules 111. That is, in this application, the input data of each processing unit 100 can be split into multiple data parts, each data part including one first sub-matrice and one second sub-matrice. These multiple data parts are used to input to multiple first processing sub-modules 111 for parallel computation. Using the above example, the first sub-matrice input to each first processing sub-module 111 can be a 2×1 matrix, and the second sub-matrice can be a 1×4 matrix.

[0042] In this application, each first processing submodule 111 may include: a plurality of first registers, a plurality of second registers, and a plurality of first multipliers, wherein, Each first register is used to store and broadcast at least a portion of the elements of the first submatrix, based on the first actual bit width of the input matrix A as indicated by the data precision.

[0043] For example, each first register is configured to: store and broadcast a row element in a first submatrix when the first actual bit width of the input matrix A indicated by the data precision matches the first reference bit width indicated by the reference precision; and when the first actual bit width of the input matrix A indicated by the data precision does not match the first reference bit width indicated by the reference precision, such as when the first actual bit width is an even multiple of the first reference bit width, store and broadcast bits of the first reference bit width in a row element in the first submatrix, wherein multiple first registers associated with a row element store different bits in that row element.

[0044] Still with Figure 3 Taking the example in the example, expanding TILE to 32, which is the calculation flow of the first processing module 110. For example, taking a reference precision of A8W4 and a first reference bit width of 8 as an example, the first processing submodule 111 may include, as follows: Figure 4 The two 8-bit first registers A00 and A01 shown are broadcast to the two rows of the first submatrix, respectively. If the first actual bit width of each element in the input matrix A is 8 bits, then A00 and A01 store two elements in the first submatrix. If the first actual bit width of each element in the input matrix A is 16 bits, then A00 and A01 store the high 8 bits and low 8 bits of one element in the first submatrix, respectively.

[0045] Each second register is used to store and broadcast at least a portion of the elements of the second submatrix, based on the second actual bit width of the weight matrix W indicated by the data precision.

[0046] For example, each second register is configured to: store and broadcast a column of elements in the second submatrix when the second actual bit width of the weight matrix W indicated by the data precision matches the second reference bit width indicated by the reference precision; and when the second actual bit width of the weight matrix W indicated by the data precision does not match the second reference bit width indicated by the reference precision, such as when the second actual bit width is an even multiple of the second reference bit width, store and broadcast bits of the second reference bit width in a column of elements in the second submatrix, wherein multiple second registers associated with a column of elements store different bits in that column of elements.

[0047] Still with Figure 3 Taking the example above, with a reference accuracy of A8W4 and a second reference bit width of 4, the first processing submodule 111 may include, for example, the following: Figure 4The four 4-bit second registers W00, W01, W02, and W03 shown are broadcast to the four columns of the second submatrix, respectively. If the second actual bit width of each element (weight value) in the weight matrix W is 4 bits, then the four 4-bit registers W00, W01, W02, and W03 store one weight value in the second submatrix, respectively. If the second actual bit width of each element in the weight matrix W is 8 bits, then the four 4-bit second registers W00, W01, W02, and W03 store two elements (weight values) in the second submatrix, where W00 and W02 are used to store the lower 4 bits of the two elements (weight values), and W01 and W03 are used to store the higher 4 bits of the two elements (weight values). If the bit width of each element in the weight matrix W is 16 bits, then the four bits of one element (weight value) are deployed in W00, W01, W02, and W03 in ascending order of bit width.

[0048] Each first multiplier is used to perform a multiplication operation on the received broadcast element to obtain the first intermediate output.

[0049] For example, the first processing submodule 111 may include, as follows: Figure 4 The eight first multipliers shown (M00, M01, M02, M03, M10, M11, M12, M13) output a 2×4 size partial sum array.

[0050] In summary, the first processing submodule 111 can perform targeted parallel computations on input data based on different data precisions, taking into account both computational efficiency and the accuracy and reliability of the computation results.

[0051] The second processing submodule 112 is used to perform accumulation and group quantization operations on the first intermediate outputs of multiple first processing submodules 111 to obtain the first target output.

[0052] In this embodiment of the application, the second processing submodule 112 can be used to accumulate, group and quantize, and accumulate multiple matrix calculation results of the calculation results (i.e., the first intermediate output) of multiple first processing submodules 111.

[0053] As one possible implementation, the second processing submodule 112 may include: multiple accumulators, a second multiplier corresponding to each accumulator, and an accumulation register, wherein, Each accumulator is used to accumulate the elements at the same position in the first intermediate outputs of the multiple first multipliers in the first processing submodule 111 to obtain the second intermediate output.

[0054] Each second multiplier is used to perform group quantization on the second intermediate output of the corresponding accumulator to obtain the third intermediate output.

[0055] Each accumulator register is used to perform an accumulation operation on the third intermediate output corresponding to the output of the second multiplier to obtain the first target output.

[0056] Using the example above, the second processing submodule 112 may include: Figure 5 The eight accumulators shown (e.g., tree accumulators, denoted as T00, T01, T02, T03, T10, T11, T12, and T13) are used to accumulate the elements at corresponding positions in the calculation results (i.e., the first intermediate outputs) of the 32 first processing sub-modules 111 (TILE0~TILE31); the eight second multipliers (M20, M21, M22, M23, M30, M31, M32, and M33) are used to multiply the outputs of the accumulators by the quantization coefficients, wherein... Figure 5 S0, S1, and S2 are 4-bit quantization coefficients; eight accumulator registers (ACC00, ACC01, ACC02, ACC03, ACC10, ACC11, ACC12, and ACC13) are used to add the quantization result output by the second multiplier to the accumulator registers to complete the accumulation of multiple matrix calculation results.

[0057] In summary, the first processing module 110 supports group quantization (each column in the matrix shares a quantization coefficient), which can balance computational efficiency and computational throughput.

[0058] The second processing module 120 is used to perform a second target operation on the first target output that matches the data precision, to obtain the second target output; wherein the second target operation includes column compression operation and / or row compression operation.

[0059] It should be noted that the implementation principle of the second processing module 120 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0060] The chip in this application embodiment has multiple parallel processing modules that support multiplication, accumulation, and group quantization operations that match the precision of the input data. This can effectively balance computational efficiency and throughput while ensuring computational accuracy.

[0061] As one possible implementation method, Figure 6 This is a schematic diagram of the structure of another chip provided for an exemplary embodiment of this application.

[0062] It should be noted that the steps executed by each processing unit in the chip can be executed individually, or they can be executed together in combination with any embodiment of this application or any possible implementation in the embodiment, or they can be executed together in combination with any technical solution in related technologies. The embodiments of this application do not limit this.

[0063] like Figure 6 As shown, the chip may include multiple processing units 100 for parallel processing of input data input to each processing unit 100; wherein each processing unit 100 includes: a first processing module 110 and a second processing module 120. The first processing module 110 is used to perform a first target operation on the input data based on the data precision of the input data, and obtain a first target output; wherein the first target operation includes at least one of multiplication operation, accumulation operation, and quantization operation.

[0064] The input data is associated with the input matrix determined by the input features based on the deep learning model and the weight matrix of the deep learning model.

[0065] It should be noted that the implementation principle of the first processing module 110 can be found in the relevant description in any embodiment of this application, and will not be repeated here.

[0066] The second processing module 120 includes: a third processing submodule 121 and / or a fourth processing submodule 122, wherein, The third processing submodule 121 is used to perform column compression operation on the first target output in response to a mismatch between the second actual bit width of the weight matrix indicated by the data precision and the second reference bit width indicated by the reference precision.

[0067] The compression operation is essentially a shift-add operation. Based on the way the chip's total input data is split before input to the first processing module 120, the output of the first processing module 110 is subjected to corresponding compression or shift-add operations.

[0068] In any embodiment of this application, the number of times the column compression operation is executed (referred to as the first execution number in this application) is determined based on the ratio of the second actual bit width to the second reference bit width. For example, the first execution number = log2 (the ratio of the second actual bit width to the second reference bit width).

[0069] In this application, when the second actual bit width does not match the second reference bit width, the first number of times the column compression operation is executed can be determined based on the ratio of the second actual bit width to the second reference bit width, and the column compression operation is executed on the first target output of the first processing module 110 based on the first number of times.

[0070] In any embodiment of this application, the first target output includes stored data in multiple accumulator registers in the first processing module 110. When the first execution number of column compression operations is N2 and the second reference bit width is B2, the third processing submodule 121 is configured to: for any column compression operation in the N2 column compression operations, determine at least one first accumulator register and at least one second accumulator register associated with the same column from the multiple accumulator registers; wherein the number of bits corresponding to the same column indicated by the stored data in the first accumulator register is lower than the number of bits corresponding to the same column indicated by the stored data in the second accumulator register; after left-shifting the stored data in the second accumulator register based on B2, accumulate the stored data in the first accumulator register.

[0071] As an example, when N2 is 1, the third processing submodule 121 is used to: determine a first accumulator register and a second accumulator register associated with the same column from multiple accumulator registers; wherein, the stored data in the first accumulator register is used to indicate the low-order data in the same column. Taking a second actual bit width of 8 and a second reference bit width of B2 of 4 as an example, the stored data in the first accumulator register is used to indicate the low 4 bits in the same column, and the stored data in the second accumulator register is used to indicate the high 4 bits in the same column; in this application, the stored data in the second accumulator register can be shifted left by B2 bits and then added to the stored data in the first accumulator register.

[0072] As another example, when N2 is 2, the third processing submodule 121 is used to: for the first column compression operation, determine a first accumulator register and a second accumulator register associated with the same column from multiple accumulator registers; wherein the bit position in the same column indicated by the stored data in the first accumulator register is lower than the bit position in the same column indicated by the stored data in the second accumulator register. In this application, the stored data in the second accumulator register can be shifted left by B2 bits and then accumulated with the stored data in the first accumulator register.

[0073] For the second column compression operation, a first accumulator register and a second accumulator register associated with the same column can be determined from multiple accumulator registers. The data stored in the first accumulator register is used to indicate the low-order data of the same column, and the data stored in the second accumulator register is used to indicate the high-order data of the same column. In this application, the data stored in the second accumulator register can be shifted left by 2*B2 bits and then accumulated with the data stored in the first accumulator register.

[0074] For example, the internal structure of the second processing module 120 may be as follows: Figure 7As shown, it mainly includes the following two parts: column compression (referred to as the third processing submodule in this application) and row compression (referred to as the fourth processing submodule in this application). Taking the reference precision as A8W4 as an example, column compression can adapt to three data formats: no compression to W4, 2 columns compressed into 1 column to adapt to W8, and 4 columns compressed into 1 column to adapt to W16. Similarly, row compression can adapt to two data formats: no compression to A8 and 2 rows compressed into 1 row to adapt to A16.

[0075] It should be understood that, Figure 7 The example only uses the data precision or data type (Data_type) of the input data to support any pairwise combination of A8 (8bit), A16 (16bit) and W4 (4bit), W8 (8bit), W16 (16bit), etc., but this application is not limited to this. For example, the data precision or data type of the input data can support A8 (8bit), A16 (16bit), A32 (8bit), A64 (bit), etc., as well as W8 (8bit), W16 (16bit), W32 (8bit), W64 (bit), etc. The embodiments of this application do not limit this.

[0076] Figure 7 In column compression, for the first column compression operation: it is necessary to remove the high-order data (such as...) Figure 5 The data is shifted left by 4 bits and then the low-order data (ACC00) is added. For the second column compression operation, the high-order data needs to be shifted left by 8 bits and then the low-order data is added. Here, the high-order data is the stored data (or compression result) of ACC02 and ACC03, and the low-order data is the stored data (or compression result) of ACC00 and ACC01. For example, the compression result of ACC02 is shifted left by 8 bits and then the compression result of ACC00 is added. The compression result of ACC03 is shifted left by 8 bits and then the compression result of ACC01 is added.

[0077] The fourth processing submodule 122 is used to perform a row compression operation on the first target output in response to a mismatch between the first actual bit width of the input matrix indicated by the data precision and the first reference bit width indicated by the reference precision.

[0078] In any embodiment of this application, the number of times the compression operation is executed (referred to as the second execution number in this application) is determined based on the ratio of the first actual bit width to the first reference bit width. For example, the second execution number = log2 (the ratio of the first actual bit width to the first reference bit width).

[0079] That is, in this application, when the first actual bit width does not match the first reference bit width, the second execution number of the line compression operation can be determined according to the ratio of the first actual bit width to the first reference bit width, and the line compression operation can be performed on the first target output of the first processing module 110 or the output result of the third processing submodule 121 according to the second execution number.

[0080] For example, if the first actual bit width does not match the first reference bit width, but the second actual bit width matches the second reference bit width, a line compression operation can be performed on the first target output of the first processing module 110 according to the second execution count; and if the first actual bit width does not match the first reference bit width, and the second actual bit width does not match the second reference bit width, a line compression operation can be performed on the output result of the third processing submodule 121 according to the second execution count.

[0081] In any embodiment of this application, an example is given where the first actual bit width does not match the first reference bit width, but the second actual bit width matches the second reference bit width. The first target output includes stored data in multiple accumulator registers in the first processing module 110. When the second execution number of the row compression operation is N1 and the first reference bit width is B1, the fourth processing submodule 122 is configured to: for any row compression operation in the N1 row compression operations, determine at least one third accumulator register and at least one fourth accumulator register associated with the same row from the multiple accumulator registers; wherein the number of bits corresponding to the same row indicated by the stored data in the third accumulator register is lower than the number of bits corresponding to the same row indicated by the stored data in the fourth accumulator register; and after left-shifting the stored data in the fourth accumulator register based on B1, accumulate it with the stored data in the third accumulator register.

[0082] It should be noted that the implementation principle of row compression is similar to that of column compression, and will not be elaborated here. For example, when N1 is 1, the fourth processing submodule 122 is used to: determine a third accumulator register and a fourth accumulator register associated with the same row from multiple accumulator registers; wherein, the data stored in the third accumulator register is used to indicate the low-order data in the same row, and the data stored in the fourth accumulator register is used to indicate the high-order data in the same row. Taking a second actual bit width of 8 and a second base bit width of B2 of 4 as an example, the data stored in the third accumulator register is used to indicate the low 4 bits in the same row, and the data stored in the fourth accumulator register is used to indicate the high 4 bits in the same row. In this application, the data stored in the fourth accumulator register can be shifted left by B2 bits and then added to the data stored in the third accumulator register.

[0083] The internal structure of the second processing module 120 is still as follows: Figure 7As shown in the example, during line compression, the high-order data (REG10) can be shifted left by 8 bits and then the low-order data (REG00) can be added.

[0084] In summary, the hardware implementation of compression uses addition and concatenation. The high-order data is added to the high-order part of the low-order data, and then the lower 4 or 8 bits of the low-order data are directly concatenated to the end of the addition result, thus completing one compression operation.

[0085] In any embodiment of this application, the second processing module 120 may further include a conversion submodule, wherein the conversion submodule is used to convert the data type of the second target output to the target data type.

[0086] The target data type is the data type required in practice, including but not limited to: FP32 (32-bit floating-point number), INT32 (signed 32-bit integer), etc.

[0087] In summary, it is possible to convert the chip's output into different data types, thereby improving application flexibility and meeting the personalized application needs of different application scenarios.

[0088] In the chip of this application embodiment, the second processing module performs a targeted compression operation on the output result of the first processing module based on the splitting method of the input data in the first processing module, which can improve the output accuracy of the chip and meet the actual application requirements.

[0089] In any embodiment of this application, taking an input matrix A of 8×32, a weight matrix W of 32×32, and input data sizes of 2×32 (two rows in A) and 32×4 (four columns in W) input to each PE in parallel as an example, the chip structure provided in this application can be as follows: Figure 8 As shown, each PE contains 256 MAC units, and the entire tensor computation array consists of 32 PEs, containing a total of 8192 MAC units.

[0090] Each PE consists of a first processing module (hereinafter referred to as MAC_ARRAY) and a second processing module (hereinafter referred to as DATA_TYPE_COMPRESS).

[0091] The first processing unit (MAC_ARRAY) can consist of 32 first processing subunits ( Figure 4 A multiplier array labeled TILE and a second processing submodule ( Figure 5 The matrix is ​​composed of TILE_COMPRESS and performs multiplication operations between 8×32 and 32×32 matrices.

[0092] As an example, the input data size to each PE is 2×32 (two rows in A, referred to as the first part matrix in this application) and 32×4 (four columns in W, referred to as the second part matrix in this application). Taking the input to each TILE as an example, the size is 2×1 (referred to as the first sub-matrix in this application, i.e., one column in the first part matrix) and 1×4 (referred to as the second sub-matrix in this application, i.e., one row in the second part matrix). Each PE includes 32 TILEs, that is, the TILEs are expanded to 32, which is the calculation flow of the first processing module (MAC_ARRAY).

[0093] For example, taking the outer product of two vectors, 2×1 and 1×4, deployed in each TILE, the matrix multiplication calculation process for each TILE can be as follows: Figure 3 As shown, each TILE calculates a partial sum array of size 2×4. The results of the four TILE calculations are summed to complete the matrix multiplication of 2×4 and 4×4.

[0094] For example, taking TILE0 as an example, the two 8-bit first registers A00 and A01 are broadcast to the two rows of the first submatrix respectively. If the bit width of each element in the input matrix A is 8 bits, then A00 and A01 store two elements in the first submatrix. If the bit width of each element in the input matrix A is 16 bits, then A00 and A01 store the high 8 bits and low 8 bits of one element in the first submatrix respectively.

[0095] Four 4-bit second registers W00, W01, W02, and W03 are used to store weight values ​​and are broadcast to the four columns of the second submatrix respectively. If the bit width of each element (weight value) in the weight matrix W is 4 bits, then the four 4-bit registers W00, W01, W02, and W03 store one weight value in the second submatrix respectively. If the bit width of each element in the weight matrix W is 8 bits, then the four 4-bit registers W00, W01, W02, and W03 store two elements (weight values) in the second submatrix respectively. Among them, W00 and W02 are used to store the lower 4 bits of the two weight values, and W01 and W03 are used to store the higher 4 bits of the two weight values. If the bit width of each element in the weight matrix W is 16 bits, then the four bits of one element (weight value) are deployed in W00, W01, W02, and W03 in ascending order of low to high.

[0096] Ultimately, each TILE's eight multipliers output a 2×4 size partial sum array.

[0097] Figure 5In this module, the second processing submodule (TILE_COMPRESS) is responsible for accumulating, grouping, quantizing, and accumulating multiple matrix calculation results from the 32 TILE calculations. Specifically, T00, T01, T02, T03, T10, T11, T12, and T13 are eight accumulators (such as tree accumulators), responsible for accumulating the corresponding elements of the 32 TILE arrays. In the A8W4 format, the eight tree accumulators directly output the matrix multiplication result without subsequent row and column compression operations. In other formats, such as A16W4, A8W8, A16W8, A8W16, and A16W16, the output of the tree accumulators is a partial sum array, which needs to be compressed by the second processing module (DATA_TYPE_COMPRESS) before the final matrix multiplication result can be output.

[0098] The first processing module (MAC_ARRAY) supports group quantization, meaning that each column of the array shares a 4-bit quantization coefficient. M20, M21, M22, M23, M30, M31, M32, and M33 are eight second multipliers responsible for multiplying the matrix multiplication result with the quantization coefficient.

[0099] ACC00, ACC01, ACC02, ACC03, ACC10, ACC11, ACC12, and ACC13 are storage registers, which are reused as accumulator registers in the first processing module (MAC_ARRAY). The quantization result and the accumulator register are added together by eight adders (A00, A01, A02, A03, A10, A11, A12, and A13) to complete the accumulation of multiple matrix calculation results.

[0100] As an example, the structure of the second processing module (DATA_TYPE_COMPRESS) can be as follows: Figure 7 As shown, its main function is to compress the stored data in ACCREG (accumulator register array, including the aforementioned accumulator registers ACC0~ACC13).

[0101] In other words, the data precision or data type (Data_type) of the input data supports any pairwise combination of A8 (8bit), A16 (16bit) and W4 (4bit), W8 (8bit), W16 (16bit). Therefore, the output of the first processing module (MAC_ARRAY) needs to be compressed in rows and / or columns to adapt to different data formats.

[0102] The internal structure of the second processing module (DATA_TYPE_COMPRESS) mainly consists of two parts: a column compression unit (referred to as the third processing submodule in this application) and a row compression unit (referred to as the fourth processing submodule in this application). Column compression can adapt to three data formats: W4 without compression, W8 by compressing 2 columns into 1 column, and W16 by compressing 4 columns into 1 column. Similarly, row compression can adapt to two data formats: A8 without compression and A16 by compressing 2 rows into 1 row.

[0103] As one possible implementation, compression is essentially a shift-add operation. Based on the way the chip's total input data is split before input to the first processing module (MAC_ARRAY), corresponding shifting is required during compression. In column compression, the first column compression operation involves shifting the high-order data (ACC01) left by 4 bits and adding the low-order data (ACC00). The second column compression operation involves shifting the high-order data left by 8 bits and adding the low-order data. Here, the high-order data is the compression result of ACC02 and ACC03, and the low-order data is the compression result of ACC00 and ACC01. Similarly, in row compression, the high-order data (REG10) is shifted left by 8 bits and added to the low-order data (REG00). The hardware implementation of compression uses addition and concatenation. The high-order data is added to the high-order portion of the low-order data, and then the lower 4 or 8 bits of the low-order data are directly concatenated to the end of the addition result, thus completing one compression operation.

[0104] As an example, the structure for reading from ACCREG and sending it to tightly coupled memory (TCM, a type of memory designed for high-performance, low-latency data access) or other computing units can be as follows: Figure 9 As shown, for ease of explanation, Figure 9 The ACCREG values ​​from all PEs are aggregated and abstracted into an 8×32 accumulator register array (32-bit). ACCREG supports row-by-row or column-by-column reading, reading one row or four columns of data per clock cycle. After selection by selector MUX0, the matrix read data is output. The output of the matrix read data supports selectable INT32 or FP32 precision. The INT32 and FP32 results are selected by selector MUX1 before output.

[0105] In summary, the solution provided in this application has at least the following advantages: 1. Supports mixed-precision matrix calculation. The basic calculation unit of the array is A8W4. This application can combine the basic A8W4 array into A8W8, A8W16, A16W4, A16W8 and A16W16 arrays. At the same time, the splicing logic reduces dynamic power consumption after accumulation.

[0106] 2. It supports group quantization, balancing computational accuracy and computational throughput.

[0107] 3. The accumulator register array ACCREG is bound to the computation logic, which reduces power density and supports flexible reading. The result can be selected with INT32 or FP32 precision.

[0108] To implement the above embodiments, this application also proposes an electronic device including the chip described in any of the foregoing embodiments.

[0109] Figure 10 This is a schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. For example, the electronic device 1000 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0110] Reference Figure 10 The electronic device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0111] Processing component 1002 typically controls the overall operation of electronic device 1000, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1002 may include one or more processors 1020 to execute instructions to complete all or part of the steps performed by the modules or units in the chip described above. Furthermore, processing component 1002 may include one or more modules to facilitate interaction between processing component 1002 and other components. For example, processing component 1002 may include a multimedia module to facilitate interaction between multimedia component 1008 and processing component 1002.

[0112] Memory 1004 is configured to store various types of data to support the operation of electronic device 1000. Examples of this data include instructions for any application or method operating on electronic device 1000, contact data, phonebook data, messages, pictures, videos, etc. Memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0113] Power component 1006 provides power to various components of electronic device 1000. Power component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 1000.

[0114] Multimedia component 1008 includes a screen that provides an output interface between the electronic device 1000 and the user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1008 includes a front-facing camera and / or a rear-facing camera. When the electronic device 1000 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0115] Audio component 1010 is configured to output and / or input audio signals. For example, audio component 1010 includes a microphone (MIC) configured to receive external audio signals when electronic device 1000 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1004 or transmitted via communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio signals.

[0116] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, start buttons, and lock buttons.

[0117] Sensor assembly 1014 includes one or more sensors for providing state assessments of various aspects of electronic device 1000. For example, sensor assembly 1014 may detect the on / off state of electronic device 1000, the relative positioning of components such as the display and keypad of electronic device 1000, changes in position of electronic device 1000 or a component of electronic device 1000, the presence or absence of user contact with electronic device 1000, the orientation or acceleration / deceleration of electronic device 1000, and temperature changes of electronic device 1000. Sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1014 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 1014 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0118] Communication component 1016 is configured to facilitate wired or wireless communication between electronic device 1000 and other devices. Electronic device 1000 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 1016 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1016 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), Bluetooth, and other technologies.

[0119] In an exemplary embodiment, the electronic device 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the steps performed by the modules or units in the aforementioned chip.

[0120] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, which can be executed by a processor 1020 of an electronic device 1000 to complete the steps performed by the modules or units in the chip. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0121] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0122] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0123] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0124] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and compact disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0125] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0126] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0128] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A chip, characterized in that, include: Multiple processing units are configured to perform parallel processing on input data input to each processing unit; wherein any one of the processing units includes: a first processing module and a second processing module; The first processing module is configured to perform a first target operation on the input data based on the data precision to which the input data belongs, and obtain a first target output; wherein the first target operation includes at least one of multiplication operation, accumulation operation, and quantization operation; The second processing module is used to perform a second target operation on the first target output that matches the data precision, to obtain a second target output; wherein the second target operation includes column compression operation and / or row compression operation.

2. The chip according to claim 1, characterized in that, The first processing module includes multiple first processing sub-modules that process in parallel, and a second processing sub-module, wherein... The first processing submodule is used to perform a multiplication operation on the input data that matches the precision of the data to obtain a first intermediate output; The second processing submodule is used to perform accumulation and group quantization operations on the first intermediate outputs of multiple first processing submodules to obtain the first target output.

3. The chip according to claim 2, characterized in that, The input data includes multiple first sub-matrices in the input matrix determined based on the input features of the deep learning model and multiple second sub-matrices in the weight matrix of the deep learning model; The first processing submodule includes: multiple first registers, multiple second registers, and multiple first multipliers, wherein, The first register is used to store and broadcast at least a portion of the elements of the first submatrix according to the first actual bit width of the input matrix indicated by the data precision; The second register is used to store and broadcast at least a portion of the elements of the second submatrix according to the second actual bit width of the weight matrix indicated by the data precision; The first multiplier is used to perform a multiplication operation on the received broadcast elements to obtain the first intermediate output.

4. The chip according to claim 3, characterized in that, The first register is used for: In response to the first actual bit width matching the first reference bit width, store and broadcast one row element in the first submatrix; In response to the first actual bit width being an even multiple of the first reference bit width, the bits of the first reference bit width in the row element are stored and broadcast. The first registers associated with the row element store different bits in the row element.

5. The chip according to claim 4, characterized in that, The second register is used for: In response to the second actual bit width matching the second reference bit width, a column of elements in the second submatrix is ​​stored and broadcast; In response to the second actual bit width being an even multiple of the second reference bit width, the bits of the second reference bit width in the column of elements are stored and broadcast; The plurality of second registers associated with the column of elements store different bits in the column of elements.

6. The chip according to claim 2, characterized in that, The second processing submodule includes: multiple accumulators, a second multiplier corresponding to each accumulator, and an accumulation register, wherein, The accumulator is used to accumulate the elements at the same position in the first intermediate outputs of the multiple first multipliers in the first processing submodule to obtain the second intermediate output. The second multiplier is used to perform a group quantization operation on the second intermediate output of the corresponding accumulator to obtain the third intermediate output; The accumulator register is used to perform an accumulation operation on the third intermediate output corresponding to the output of the second multiplier to obtain the first target output.

7. The chip according to claim 1, characterized in that, The second processing module is used for: In response to the mismatch between the data precision and the reference precision, a second target operation matching the data precision is performed on the first target output to obtain the second target output.

8. The chip according to claim 7, characterized in that, The input data is associated with the input matrix determined by the input features based on the deep learning model and the weight matrix of the deep learning model; The second processing module includes: a third processing submodule and / or a fourth processing submodule, wherein, The third processing submodule is configured to perform the column compression operation on the first target output in response to a mismatch between the second actual bit width of the weight matrix indicated by the data precision and the second reference bit width indicated by the reference precision. The fourth processing submodule is configured to perform the row compression operation on the first target output in response to a mismatch between the first actual bit width of the input matrix indicated by the data precision and the first reference bit width indicated by the reference precision.

9. The chip according to claim 8, characterized in that, The third processing submodule is used for: In response to a mismatch between the second actual bit width and the second reference bit width, the first number of executions of the column compression operation is determined based on the ratio of the second actual bit width to the second reference bit width. Based on the first number of executions, perform the column compression operation on the first target output.

10. The chip according to claim 9, characterized in that, The first target output includes the stored data in multiple accumulator registers in the first processing module; In response to the first execution count being N2 and the second reference bit width being B2, the third processing submodule is configured to: For any column compression operation in N2 column compression operations, at least one first accumulator register and at least one second accumulator register associated with the same column are determined from the plurality of accumulator registers; wherein the number of bits corresponding to the same column indicated by the stored data in the first accumulator register is lower than the number of bits corresponding to the same column indicated by the stored data in the second accumulator register. After left-shifting the stored data in the second accumulator register based on B2, the stored data in the first accumulator register is added.

11. The chip according to claim 8, characterized in that, The fourth processing submodule is used for: In response to the mismatch between the first actual bit width and the first reference bit width, the second number of executions of the row compression operation is determined based on the ratio of the first actual bit width to the first reference bit width. The line compression operation is performed on the first target output based on the second number of executions.

12. The chip according to claim 11, characterized in that, The first target output includes the stored data in multiple accumulator registers in the first processing module; In response to the second execution count being N1 and the first reference bit width being B1, the fourth processing submodule is configured to: For any one of the N1 line compression operations, at least one third accumulation register and at least one fourth accumulation register associated with the same line are determined from the plurality of accumulation registers; wherein the number of bits corresponding to the same line indicated by the stored data in the third accumulation register is lower than the number of bits corresponding to the same line indicated by the stored data in the fourth accumulation register. Based on the left shift of the stored data in the fourth accumulator register according to B1, the data is then accumulated with the stored data in the third accumulator register.

13. The chip according to any one of claims 7-12, characterized in that, The second processing module is further configured to: In response to the data precision matching the benchmark precision, the first target output is used as the second target output.

14. The chip according to any one of claims 1-12, characterized in that, The second processing module further includes: The conversion submodule is used to convert the data type of the second target output to the target data type.

15. An electronic device, characterized in that, Includes the chip as described in any one of claims 1-14.

Images