AI calculation method and system

Abstract

One or more embodiments of the invention provide an AI calculation method and system. According to the method, convolution operation is realized by setting a short instruction set, and the method comprises the following steps of: loading data of a local area in a feature map in parallel by utilizing a plurality of general registers; utilizing the plurality of parameter registers to load network parameters; respectively multiplying the data in each parameter register by the data in the general register at the corresponding position in parallel, respectively loading multiplication results into different accumulation registers, accumulating the loaded data by the accumulation registers, and storing the accumulated data in the general register; the corresponding position is a position which is multiplied and accumulated with a network parameter in a process of executing convolution by using a sliding window; and generating corresponding elements in the output feature map according to an accumulation result in the accumulation register. According to the scheme, on the basis that the flexible and configurable advantages of a general processor are reserved, AI calculation is accelerated on hardware, and meanwhile hardware overhead is reduced.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer technology, and in particular, to an AI computing method and system. Background Art

[0002] In recent years, edge AI computing has benefited from its real-time and stability advantages and has attracted the attention of more and more emerging or traditional application fields. In the low-power application of edge AI computing, the current solutions are mainly divided into two types. One is to calculate lightweight AI algorithms with the help of traditional general-purpose computing chips (such as CPU, MCU, etc.), and the other is to integrate dedicated AI acceleration chips / modules on PCB boards or SoC chips. Although the first solution does not require additional hardware costs and has strong programmable flexibility, its disadvantages are high power consumption, poor real-time performance, or poor algorithm accuracy; although the second solution has better accuracy due to lower power consumption and higher computing power, its disadvantages are that it adds more hardware costs and supports limited AI algorithms, and dedicated deeply customized hardware is difficult to be compatible with new AI algorithms. Summary of the invention

[0003] In view of this, one or more embodiments of this specification provide an AI computing method and system.

[0004] According to a first aspect of one or more embodiments of this specification, an AI computing method is proposed, which implements convolution operation by setting a short instruction set, and the method includes:

[0005] Using multiple general registers to load various data of local areas in the feature map in parallel;

[0006] Load network parameters using multiple parameter registers;

[0007] The data in each parameter register is multiplied in parallel with the data in the general register at the corresponding position, and the multiplication results are loaded into different accumulation registers, and the accumulation registers accumulate the loaded data. The corresponding position is the position where the network parameters are multiplied and accumulated during the convolution process using the sliding window;

[0008] Generate corresponding elements in the output feature map according to the accumulated results in the accumulation register.

[0009] In some embodiments, the plurality of general registers are divided into M groups, and the method of using the plurality of general registers to load the respective data of the local area in the feature map in parallel includes:

[0010] A general register in each of the M groups is used to load data of a local area in the feature map.

[0011] In some embodiments, the set short instruction set includes pre-set multiplication and accumulation instructions corresponding to various modes of accessing general registers, and the method further includes:

[0012] Determine a mode of accessing the general register according to the group to which the general register for loading each data in the local area belongs;

[0013] The multiplication and accumulation instruction corresponding to the mode is called to perform multiplication and accumulation on the data in the local area and the network parameters.

[0014] In some embodiments, the method further comprises:

[0015] For each multiply-accumulate instruction, the address of the accumulation register is encoded according to the accumulation register into which each packet can be loaded.

[0016] In some embodiments, generating a corresponding element in the output feature map according to the accumulated result in the accumulation register includes:

[0017] After the accumulation register completes the accumulation of the characteristic graph data, outputting the accumulation result and clearing the accumulation register;

[0018] Generate a corresponding element in the output feature map according to the accumulation result output by the accumulation register.

[0019] According to a second aspect of one or more embodiments of this specification, an AI computing system is proposed, including:

[0020] A data storage unit, used for storing feature map data and network parameters;

[0021] An instruction decoding unit, used for decoding instructions in a set short instruction set;

[0022] The instruction execution unit is used to control the register array to perform convolution operation according to the decoding result of the instruction decoding unit; the register array includes:

[0023] A plurality of general registers for loading various data of local areas in the feature map in parallel;

[0024] Multiple parameter registers for loading network parameters;

[0025] Multiple accumulator registers are used to load the data in each parameter register and the multiplication result of the data in the general register at the corresponding position respectively, and accumulate the loaded data, and the corresponding position is the position where the network parameters are multiplied and accumulated during the convolution process using the sliding window.

[0026] In some embodiments, the plurality of general registers are divided into M groups, and a general register in each group is loaded with data of a local region in the feature map.

[0027] In some embodiments, the set short instruction set includes pre-set multiplication and accumulation instructions corresponding to various modes of accessing general registers, and the instruction execution unit is used to:

[0028] Determine a mode of accessing the general register according to the group to which the general register for loading each data in the local area belongs;

[0029] The multiplication and accumulation instruction corresponding to the mode is called to perform multiplication and accumulation on the data in the local area and the network parameters.

[0030] In some embodiments, the instruction execution unit is used to:

[0031] For each multiply-accumulate instruction, the address of the accumulation register is encoded according to the accumulation register into which each packet can be loaded.

[0032] In some embodiments, when the instruction execution unit generates a corresponding element in the output feature map according to the accumulated result in the accumulation register, it is specifically used to:

[0033] After the accumulation register completes the accumulation of the characteristic graph data, outputting the accumulation result and clearing the accumulation register;

[0034] Generate a corresponding element in the output feature map according to the accumulation result output by the accumulation register.

[0035] The AI ​​calculation method proposed in the embodiment of this specification is achieved by decomposing the conventional convolution operation into multiple data loading and calculation steps, and implementing the convolution operation by setting a short instruction set. By loading each element in each local area of ​​the feature map in parallel, each convolution kernel element is multiplied with the feature map element at the corresponding position at the same time, and loaded into different accumulator registers respectively. The same accumulator register accumulates the partial products loaded therein, thereby achieving high-concurrency convolution operations and improving the speed of AI calculations. Since only the general registers in each group need to be address indexed at each degree of parallelism, the multiplication and accumulation instructions in the short instruction set can be optimized to low-bit width instructions, thereby reducing hardware overhead. Therefore, the present application scheme accelerates AI calculations in hardware while reducing hardware overhead while retaining the flexible and configurable advantages of general-purpose processors. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is an application scenario diagram of an AI computing method provided by an exemplary embodiment.

[0037] Figure 2 It is a flowchart of an AI calculation method provided by an exemplary embodiment.

[0038] Figure 3A It is a schematic diagram of loading feature map data into a general register.

[0039] Figure 3B It is a schematic diagram of parameter registers loading network parameters.

[0040] Figure 3C It is a schematic diagram of multiplying and accumulating data in a general register with data in a parameter register.

[0041] Figure 4A is a schematic diagram of the general register array.

[0042] Figure 4B It is a schematic diagram of the working mode of the general register and the working mode of the accumulator register.

[0043] Figure 5 It is a structural diagram of an AI computing system provided by an exemplary embodiment. DETAILED DESCRIPTION

[0044] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0045] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0046] In order to solve the drawbacks of two conventional processing methods in edge AI computing, namely, using general-purpose computing chips (such as CPU, MCU, etc.) to calculate lightweight AI algorithms, and integrating dedicated AI acceleration chips / modules on PCB boards or SoC chips, the embodiment of the present application proposes an AI computing method that uses a short instruction set with low bit width and high parallelism to implement convolution operations.

[0047] In deep learning algorithms represented by neural networks, the main amount of computation is concentrated in the convolution operation, and the basic operation of the convolution operation is the tensor product, and the operation formula is shown in (1):

[0048]

[0049] Among them, K n is the network parameter (convolution kernel), X m is the feature map vector, A mn Calculates the result of the tensor product (partial product).

[0050] Afterwards, the partial products are accumulated, as shown in formula (2), to obtain the convolution calculation result:

[0051] A=∑Amn (2)

[0052] Among them, A represents the result of partial accumulation.

[0053] Traditional convolution operation, for each output element, needs to calculate the product of each element of the convolution kernel and each feature map element in the corresponding window in the input feature map and add them one by one, and needs to frequently read the input feature map data from the memory, which is time-consuming and has low computational efficiency. The embodiment of the present application adopts high-parallel MAC (Multi-Accumu late, multiplication and accumulation) calculation to accelerate. However, since the high-parallel MAC calculation will cause the tensor MAC instruction to require more bit registers for address indexing, it will require a higher instruction bit width, resulting in the hardware overhead of modules such as instruction decoders being doubled. Therefore, the embodiment of the present application disassembles the convolution operation, and designs a series of short instructions for the disassembled operation, thereby obtaining a high-concurrency, low-bit-width MAC-related instruction set for performing convolution operations.

[0054] The short instruction set proposed in the embodiment of the present application is based on the basic instruction set and the extended vector instruction set. Among them, the basic instruction set can be a CPU native instruction set, such as a RISC-V instruction set; the extended vector instruction set can adopt a SIMD (Single Instruction, Multiple Data) instruction encoding mode. In addition, the embodiment of the present application proposes a corresponding microinstruction architecture based on the extended vector instruction set to identify and execute short instructions in the short multiplication and accumulation instruction set.

[0055] The following is a description of an AI calculation method provided in an embodiment of the present application.

[0056] like Figure 1 The figure shows an application scenario of the AI ​​computing method. Figure 1It includes: terminal device 10, edge box 20, and cloud server 30.

[0057] The terminal device 10 can be a device that the user directly interacts with, and has the ability to collect, preliminarily process and send data. The terminal device 40 can be a smart phone, sensor, camera, smart cockpit, etc., which is responsible for collecting environmental data or user-generated data and sending the data to the edge box 20 for processing.

[0058] The edge box 20 can be a computing device deployed at the edge of the network, which uses a set short instruction set and a corresponding microinstruction architecture to achieve efficient AI computing. The results of the AI ​​computing can be displayed locally, fed back to the terminal device 10, or transmitted to the cloud server 30.

[0059] The cloud server 30 may be a remote data center that provides powerful computing resources, storage capabilities, and model training services. The AI ​​model deployed in the edge zygote 20 may be trained in the cloud server 30 and pushed.

[0060] The description in this application only details a single terminal device 10, edge box 20, and cloud server 30, but those skilled in the art should understand that the diagrams are intended to illustrate the operations of the terminal device 10, edge box 20, and cloud server 30 involved in the technical solution of this application, and do not imply limitations on their number, type, or location, etc. It should be noted that if additional modules are added to the illustrated environment or individual modules are removed from it, the underlying concepts of the example embodiments of this application will not be changed.

[0061] It should be noted that the AI ​​calculation method proposed in this application is not only applicable to Figure 1 The application scenarios shown are also applicable to other devices with AI computing requirements.

[0062] Figure 2 A flowchart of an AI calculation method provided in an embodiment of the present application is shown, and the method includes:

[0063] Step 201, using a plurality of general registers to load various data of the local area in the feature map in parallel.

[0064] In each loading cycle, multiple general registers synchronously and respectively load a data in a local area, wherein the local area may be a window in the feature map data, and its size may be the same as the convolution kernel size.

[0065] In the embodiment of the present application, the feature map data may be stored in a buffer area of ​​the data storage unit. Figure 3A As shown, the feature map data is as follows mAs shown, the local area in the feature map is shown as a dotted box, which can be any area in the feature map. In this example, assuming that the local area contains M data, the M general registers Reg-X can be used to load the M data at the same time.

[0066] In some embodiments, the general registers used for AI calculations can be divided into M groups. Assuming that each group has N general registers, the first group contains general registers X0 to XN-1, the second group contains parameter registers XN+1 to X2N, and so on. After such grouping, a general register in each of the M groups can be used to load a data of a local area in the feature map.

[0067] For example, in a load cycle, the general registers in the same position (same row) in each group can be used, such as Figure 3A The X0, XN, X2N, and X3N shown in the figure read a data in the partial area shown by the black dotted line respectively. That is, the feature map element "x00" is loaded in X0, the feature map element "x01" is loaded in XN, the feature map element "x10" is loaded in X2N, and the feature map element "x11" is loaded in X3N. In the next loading cycle, X1, XN+1, X2N+1, and X3N+1 can be used to read the partial area shown by the black dotted line respectively, that is, the feature map element "x02" is loaded in X1, the feature map element "x03" is loaded in XN+1, the feature map element "x12" is loaded in X2N+1, and the feature map element "x13" is loaded in X3N+1.

[0068] In the embodiment of the present application, the subsequent multiplication-accumulation calculation may be performed after one loading cycle, or may be performed after multiple loading cycles.

[0069] Step 202: Load network parameters using multiple parameter registers.

[0070] The network parameter data (convolution kernel data) can be stored in the storage area of ​​the data storage unit, such as Figure 3B K n As shown in the figure, the parameter register Reg-C includes C0, C1, C2, and C3, which are used to load the convolution kernel elements "w00", "w01", "w10", and "w11" respectively.

[0071] For each load operation indicated by a short instruction, the convolution kernel data loaded by the parameter register may be complete convolution kernel data or partial convolution kernel data.

[0072] For example, when the size of the convolution kernel is 4*4, in the present application scheme, a 2*2 size can be loaded first. After the operation is completed for the 2*2 area, the next 2*2 area is loaded until the operation of all convolution kernels is completed.

[0073] Step 103, multiply the data in each parameter register in parallel with the data in the general register at the corresponding position, and load the multiplication results into different accumulation registers respectively.

[0074] Among them, the accumulator register is used to accumulate the loaded data, and the corresponding position is the position where the network parameters are multiplied and accumulated during the convolution process using the sliding window.

[0075] In traditional convolution operations, a sliding window is performed in the feature map data, and the feature map elements in the window are multiplied and accumulated with the convolution kernel elements. The accumulated result is the value of the corresponding element of the output feature map. In the sliding window process, the position where the data in the convolution kernel can be multiplied and accumulated with the data in the general register is the corresponding position of the general register.

[0076] In an embodiment of the present application, instead of multiplying and accumulating multiple elements in the convolution kernel matrix with the feature map matrix in the window at the same time, an element in the convolution kernel is used alone to perform a multiplication operation with the feature map element in the general register at the corresponding position, and the partial products obtained are stored in different accumulation registers. That is, in the calculation cycle for an element in the convolution kernel, the partial products obtained by multiplying the element with different feature map elements are not accumulated. After completing the multiplication operation and partial product loading for the element in the convolution kernel, a similar operation is performed on the element in the next convolution kernel. However, since the position of the element in the convolution kernel is different from that of the previous element, the position of the corresponding general register is also different, that is, the feature map element to be multiplied is also different, and the partial product of the convolution kernel element and the feature map element to be multiplied is loaded again in the accumulation register. Among them, the partial product corresponding to the first feature map element multiplied by the convolution kernel element is loaded in the first position of the accumulation register, and the positions of the other partial product loadings are determined according to the positional relationship between the feature map elements. The first position refers to the position where the partial product of the first feature map element is loaded when the first convolution kernel element is operated, and it is not necessarily the first position in the physical sense. Next, the other elements in the convolution kernel are multiplied with the feature map elements and partially loaded in the same way.

[0077] by Figure 3CTaking the registers shown in as an example, assuming that the multiplication and partial product loading operations start after two loading cycles, the feature map data have been loaded into the general registers x0, xN, x2N, x3N, x1, xN+1, x2N+1, and x3N+1 respectively, and the convolution kernel data have also been loaded into the parameter registers C0, C1, C2, and C3.

[0078] For the convolution kernel elements in the parameter register C0, the general registers that may perform multiplication operations include x0, xN, x1, x2N, x3N, and x2N+1. Therefore, the convolution kernel element w00 in C0 is multiplied with the feature map elements stored in these positions, and the six partial products are stored in the six accumulation registers Reg-A respectively, and the six multiplication results are not accumulated. Specifically, the partial products of the convolution kernel element w00 and the feature map elements in x0, xN, x1, x2N, x3N, and x2N+1 can be loaded in A0, A1, A2, A4, A5, and A6 respectively. It should be noted that the above loading positions are only for illustration. When the corresponding relationship between the feature map elements and the accumulation registers is established, the final calculation results are the same regardless of the specific storage positions.

[0079] For the convolution kernel element w01 in the parameter register C1, the general registers that may be multiplied include xN, x1, Xn+1, x3N, x2N+1, x3N+1, so the convolution kernel element w01 is multiplied with the feature map elements stored in these positions, and the partial products are loaded in A0, A1, A2, A4, A5, and A6 respectively. That is, when loading partial products, the first partial product is loaded in the first position of the accumulator register, and the positions of the other partial products are determined according to the positional relationship between the feature map elements.

[0080] Similarly, for the convolution kernel element w10, the general registers that may perform multiplication operations include x2N, x3N, and x2N+1. Therefore, the convolution kernel element w10 is multiplied with the feature map elements stored in these positions, and the partial products are loaded into A0, A1, and A2 respectively; for the convolution kernel element w11, the general registers that may perform multiplication operations include x3N, x2N+1, and x3N+1. Therefore, the convolution kernel element w11 is multiplied with the feature map elements stored in these positions, and the partial products are loaded into A0, A1, and A2 respectively.

[0081] Each accumulation register accumulates the partial products loaded. For example, if the following partial products are loaded into accumulation register A0: w00*x00, w01*x01, w10*x10, w11*x11, then A0 accumulates these loaded partial products; similarly, accumulation register A1 accumulates the partial products w00*x01, w01*x02, w10*x11, w11*x12 loaded therein, and so on, each accumulation register accumulates the partial products recorded therein.

[0082] After completing the multiplication and partial accumulation of the feature map elements loaded into the general register, the remaining feature map elements are loaded, and steps 201 to 203 are repeated for the newly loaded parts until all feature map elements are loaded.

[0083] In some embodiments, the multiplication and partial accumulation operations of the convolution kernel elements and the feature map elements may be performed after all the feature map elements are loaded.

[0084] In the present application, after the accumulator register is cleared, the partial products generated in the subsequent operation process can be loaded again. Since the accumulator register is reusable, a specified number of accumulator registers can be set, as long as it is ensured that the partial products generated by the convolution kernel elements to be operated have loadable accumulator registers.

[0085] Step 204: Generate a corresponding element in the output feature map according to the accumulated result in the accumulation register.

[0086] During the accumulation process of the accumulation register, if an accumulation register has completed the accumulation of feature map data, the accumulation result is output and cleared.

[0087] Among them, an accumulation register has completed the accumulation of feature map data, which means that the partial accumulation result accumulated in the accumulation register has reached the multiplication and accumulation result of all elements in a window and the convolution kernel in the conventional convolution operation. In this case, the accumulation result is output as the value of the corresponding element of the output feature map.

[0088] In the above example, after accumulating w00*x00, w01*x01, w10*x10, and w11*x11 in the accumulator register A0, it is equivalent to completing the multiplication and accumulation of the first window containing x0, xN, x2N, and x3N in the feature map data and the convolution kernel matrix w00, w01, w10, and w11, and A0 outputs the accumulation result as the value of the first element in the output feature map. After all accumulator registers have output results, that is, all windows have completed multiplication and accumulation, the calculation is completed, the feature map data output by the convolution operation is obtained, and the data is written back to the storage unit.

[0089] The AI ​​calculation method proposed in the embodiment of this specification is achieved by decomposing the conventional convolution operation into multiple data loading and calculation steps, and implementing the convolution operation by setting a short instruction set. By loading each element in each local area of ​​the feature map in parallel, each convolution kernel element is multiplied with the feature map element at the corresponding position at the same time, and loaded into different accumulator registers respectively. The same accumulator register accumulates the partial products loaded therein, thereby achieving high-concurrency convolution operations and improving the speed of AI calculations. Since only the general registers in each group need to be address indexed at each degree of parallelism, the multiplication and accumulation instructions in the short instruction set can be optimized to low-bit width instructions, thereby reducing hardware overhead. Therefore, the present application scheme accelerates AI calculations in hardware while reducing hardware overhead while retaining the flexible and configurable advantages of general-purpose processors.

[0090] When the general registers are divided into M groups and a register in each group is used to read a data in a local area of ​​the feature map, each parallelism only needs to access a set of general register combinations, so the required bit width of the instruction address for each parallelism is bits, N is the number of general registers in each group. In the conventional convolution method, the required bit width of the instruction address is bits, so in the present application scheme, each parallel instruction address saves bits. If the MAC has P parallelism, then Proportional instruction width.

[0091] In addition, when general registers are grouped, each data in a local area (window) is loaded through different groups of general registers in each loading cycle. For all local areas of feature map data, the pattern of the group to which the general registers that load each data in the local area belong can be determined. That is, when the convolution kernel K n With feature map data X m When performing tensor products and accumulations, as X m There are only H possible modes for sliding to access general registers. The value of H is the grouped data, that is, H = M, so it is necessary bits to indicate the operating mode.

[0092] If the multiplication-accumulation instructions corresponding to the various modes of accessing the general registers (i.e., the working modes) are pre-set in the extended instruction set, the working mode of the local area can be determined according to the grouping corresponding to the various data in the loaded local area, thereby determining the corresponding multiplication-accumulation instructions; by calling the multiplication-accumulation instructions corresponding to the working mode, the multiplication and accumulation of the various data in the local area and the network parameters can be realized.

[0093] by Figure 4A As an example, the register array containing 32 general registers is divided into 4 groups, each group contains registers X0, X1, X2, and X3, the second group contains registers X8, X9, X10, and X11, the third group contains registers X16, X17, X18, and X19, and the fourth group contains registers X24, X25, X26, and X27. When the array is used to load feature map data in parallel, for all regions, the groups to which the general registers that load each data belong may be as follows: Figure 4B The four cases shown are: for the local area in the black dotted box, the grouping mode is M0-M1-M2-M3 (mode 1); for the local area in the red dotted box, the grouping mode is M1-M0-M3-M2 (mode 2); for the local area in the blue dotted box, the grouping mode is M2-M3-M3-M2 (mode 3); for the local area in the yellow dotted box, the grouping mode is M3-M2-M1-M0 (mode 4). The grouping modes corresponding to all other local areas in the feature map belong to one of these four modes.

[0094] Therefore, corresponding multiplication-accumulation MAC instructions can be set for these four modes. For example, for mode 1, instruction cnv16q with number 0123 can be set, for mode 2, instruction cnv16q with number 1032 can be set, for mode 3, instruction cnv16q with number 2301 can be set, and for mode 4, instruction cnv16q with number 3210 can be set. In this way, in the process of multiplication-accumulation, the corresponding instruction can be called to perform multiplication-accumulation according to the grouping mode corresponding to the local area.

[0095] In an embodiment of the present application, by splitting the long instructions for performing conventional convolution operations into multiple short instructions, and combining the computational characteristics of operators such as convolution and full connection in deep learning, the short instruction set is trimmed and unnecessary flexibility is discarded, thereby reducing the types of short instructions and reducing the instruction bit width of the MAC instruction group, while retaining the advantage of flexible instruction compilation.

[0096] In some embodiments, for each multiply-accumulate instruction, the address of the accumulation register may be encoded according to the accumulation register to which each group can be loaded, so as to further optimize the bit width of the instruction.

[0097] See also Figure 4B, assuming that for the multiplication and accumulation instruction "cnv16q 0123", the corresponding grouping mode is M0, M1, M2, M3. For the M0 grouping, the partial product of the feature map element multiplied by any convolution kernel element can only be loaded into A0 or A2, so in this group, only 2 bits of bit width are needed to encode the address of the accumulation register; for the M1 grouping, the partial product of the feature map element multiplied by any convolution kernel element can only be loaded into A1 or A3, so in this group, only 2 bits of bit width are needed to encode the address of the accumulation register; for the M2 grouping, the partial product of the feature map element multiplied by any convolution kernel element can only be loaded into A4 or A6, so in this group, only 2 bits of bit width are needed to encode the address of the accumulation register; for the M3 grouping, the partial product of the feature map element multiplied by any convolution kernel element can only be loaded into A5 or A7, so in this group, only 2 bits of bit width are needed to encode the address of the accumulation register. That is, for the multiply-accumulate instruction, a total of 8 bits are required to encode the address of the accumulation register.

[0098] Figure 4B The possibility of the accumulation register that each group can be loaded into for other multiply-accumulate instructions is also shown in FIG. It can be seen that in this example, all multiply-accumulate instructions require at most 8 bits for encoding the address of the accumulation register.

[0099] By optimizing the bit width required for encoding the accumulator register address, the bit width of the MAC instruction is further reduced.

[0100] Figure 5 This is a block diagram of an AI computing system provided by an exemplary embodiment, that is, a circuit structure diagram of a programmable AI acceleration micro-architecture. Figure 5 As shown, the system includes: a data storage unit 501, an instruction decoding unit 502, an instruction execution unit 503 and a register array 504.

[0101] The data storage unit 501 is used to store feature map data and network parameters. The data storage unit can be divided into a storage area and a cache area. The storage area is used to store instructions or parameters, and the cache area is used to store cache data, such as feature map data, and the data storage unit supports distributed storage.

[0102] The instruction decoding unit 502 is used to decode instructions in a set short instruction set, which includes a basic instruction set and an extended instruction set.

[0103] The instruction execution unit 503 is used to control the register array to perform convolution operation according to the decoding result of the instruction decoding unit.

[0104] In some embodiments, the instruction execution unit 503 includes an address generation unit 5031 , a scalar arithmetic logic unit 5032 , a tensor multiplication and accumulation unit 5033 , and a vector arithmetic logic unit 5034 .

[0105] The address generation unit 5031 is responsible for completing the initialization and update operations of the pointer address; the scalar arithmetic logic unit 5032 is responsible for various arithmetic logic calculations of scalar data, including but not limited to addition / subtraction / multiplication / division / shift / truncation / and / or, etc.; the tensor multiplication and accumulation unit 5033 is responsible for accelerating the multiplication and accumulation MAC calculation, and is composed of multiple multiplier arrays; the vector arithmetic logic unit 5034 is responsible for parallel acceleration of arithmetic logic calculations, which may include but not limited to addition / subtraction / multiplication / division / shift / truncation / and / or, etc., with the purpose of parallel acceleration of arithmetic logic calculations that need to be accelerated, while supporting non-MAC AI operators such as pooling / activation functions / element-level operations, while matching the computing speed of MAC calculations to avoid becoming a performance bottleneck.

[0106] The register array 504 includes a plurality of general registers 5041 , a plurality of parameter registers 5042 , and a plurality of accumulator registers 5043 .

[0107] Multiple general registers 5041 are used to load various data of local areas in the feature map in parallel; multiple parameter registers 5042 are used to load network parameters; multiple accumulator registers 5043 are used to load the data in each parameter register and the multiplication result of the data in the general register at the corresponding position in parallel, and accumulate the loaded data, and the corresponding position is the position where the network parameters are multiplied and accumulated during the convolution process using a sliding window.

[0108] The AI ​​computing system may also include a bus transfer unit 505, which connects the data storage unit and each register group in the processor core and is responsible for completing the data transfer function between registers and memories and between registers. In addition, the bus transfer unit is also responsible for completing operations such as data alignment and data bit width fusion of the data bus.

[0109] In some embodiments, a plurality of general registers may be divided into M groups, and a general register in each group is loaded with data of a local region in the feature map.

[0110] In some embodiments, a short instruction set is set to include pre-set multiplication and accumulation instructions corresponding to various modes of accessing general registers, and the instruction execution unit 503 is used to: determine the mode of accessing the general registers according to the group to which the general registers for loading various data in the local area belong; call the multiplication and accumulation instructions corresponding to the mode to multiply and accumulate the data in the local area and the network parameters.

[0111] In some embodiments, the instruction execution unit 503 may also be configured to: for each multiply-accumulate instruction, perform address encoding on the accumulation register according to the accumulation register to which each group can be loaded.

[0112] In some embodiments, the instruction execution unit 503 is specifically used to generate the corresponding element in the output feature map according to the accumulated result in the accumulator register: output the accumulated result and clear the accumulator register after the accumulator register completes the accumulation of the feature map data; generate the corresponding element in the output feature map according to the accumulated result output by the accumulator register

[0113] It should be noted that, in this specification, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "includes a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0114] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0115] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0116] It should be understood that although the terms first, second, third, etc. may be used to describe various information in one or more embodiments of this specification, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0117] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. An AI calculation method, characterized in that: The convolution operation is realized by setting a short instruction set, and the method comprises: Using multiple general registers to load various data of local areas in the feature map in parallel; Load network parameters using multiple parameter registers; The data in each parameter register is multiplied in parallel with the data in the general register at the corresponding position, and the multiplication results are loaded into different accumulation registers, and the accumulation registers accumulate the loaded data. The corresponding position is the position where the network parameters are multiplied and accumulated during the convolution process using the sliding window; Generate corresponding elements in the output feature map according to the accumulated results in the accumulation register.

2. The method according to claim 1, characterized in that The plurality of general registers are divided into M groups, and the plurality of general registers are used to load the data of the local area in the feature map in parallel, including: A general register in each of the M groups is used to load data of a local area in the feature map.

3. The method according to claim 2, characterized in that The set short instruction set includes pre-set multiplication and accumulation instructions corresponding to various modes of accessing general registers, and the method further includes: Determine a mode of accessing the general register according to the group to which the general register for loading each data in the local area belongs; The multiplication and accumulation instruction corresponding to the mode is called to perform multiplication and accumulation on the data in the local area and the network parameters.

4. The method according to claim 3, characterized in that Also includes: For each multiply-accumulate instruction, the address of the accumulation register is encoded according to the accumulation register into which each packet can be loaded.

5. The method according to claim 1, characterized in that Generating a corresponding element in the output feature map according to the accumulated result in the accumulation register includes: After the accumulation register completes the accumulation of the characteristic graph data, outputting the accumulation result and clearing the accumulation register; Generate a corresponding element in the output feature map according to the accumulation result output by the accumulation register.

6. An AI computing system, characterized in that: include: A data storage unit, used for storing feature map data and network parameters; An instruction decoding unit, used for decoding instructions in a set short instruction set; The instruction execution unit is used to control the register array to perform convolution operation according to the decoding result of the instruction decoding unit; the register array includes: A plurality of general registers for loading various data of local areas in the feature map in parallel; Multiple parameter registers for loading network parameters; Multiple accumulator registers are used to load the data in each parameter register and the multiplication result of the data in the general register at the corresponding position respectively, and accumulate the loaded data, and the corresponding position is the position where the network parameters are multiplied and accumulated during the convolution process using the sliding window.

7. The system according to claim 6, characterized in that The plurality of general registers are divided into M groups, and a general register in each group is loaded with data of a local area in the feature map.

8. The system according to claim 7, characterized in that The set short instruction set includes pre-set multiplication and accumulation instructions corresponding to various modes of accessing general registers, and the instruction execution unit is used for: Determine a mode of accessing the general register according to the group to which the general register for loading each data in the local area belongs; The multiplication and accumulation instruction corresponding to the mode is called to perform multiplication and accumulation on the data in the local area and the network parameters.

9. The system according to claim 8, characterized in that The instruction execution unit is used for: For each multiply-accumulate instruction, the address of the accumulation register is encoded according to the accumulation register into which each packet can be loaded.

10. The system according to claim 6, characterized in that The instruction execution unit is specifically used to generate a corresponding element in the output feature map according to the accumulated result in the accumulation register: After the accumulation register completes the accumulation of the characteristic graph data, outputting the accumulation result and clearing the accumulation register; Generate a corresponding element in the output feature map according to the accumulation result output by the accumulation register.

Images