Data interaction method and device based on extension instruction and storage medium

Abstract

The invention provides a data interaction method and device based on an extension instruction, equipment and a storage medium, and relates to the technical field of artificial intelligence. The data interaction method based on the extension instruction is applied to a heterogeneous intelligent chip, the heterogeneous intelligent chip comprises a processor and an intelligent acceleration engine, and the method comprises the following steps: obtaining a preset first extension instruction; and based on the first extension instruction, configuring the processor and the intelligent acceleration engine for data interaction between the processor and the intelligent acceleration engine. According to the embodiment of the invention, direct and rapid data interaction between the processor and the intelligent acceleration engine can be realized through the self-defined extension instruction, and the calculation speed and efficiency of the heterogeneous intelligent chip are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a data interaction method, device, and storage medium based on extended instructions. Background Technology

[0002] To meet the requirements of flexibility and computing speed in intelligent computing, intelligent chips typically adopt a heterogeneous architecture, namely a processor + intelligent acceleration engine. The processor is primarily responsible for scheduling and managing the intelligent computing process and handling some complex operator calculations; the intelligent acceleration engine handles high-concurrency simple operations, such as matrix multiplication and convolution.

[0003] To achieve high-speed and efficient computing in heterogeneous smart chips, it is usually necessary to configure the smart acceleration engine to enable the processor to control the working mode and workflow of the smart acceleration engine, so that the smart acceleration engine can focus on the computation of data streams.

[0004] Traditional methods of configuring the intelligent acceleration engine via the processor require the use of the processor's internal system bus (such as AXI / AHB / APB) and the on-chip system bus (NoC). Due to the long latency of the processor's internal system bus path and the arbitration effect of the on-chip system bus, the latency of configuring the intelligent acceleration engine is uncertain, severely impacting the interaction efficiency between the processor and the intelligent acceleration engine. Summary of the Invention

[0005] According to one aspect of the present invention, a data interaction method based on extended instructions is provided, applied to a heterogeneous intelligent chip, the heterogeneous intelligent chip including a processor and an intelligent acceleration engine, comprising: acquiring a preset first extended instruction; and configuring the processor and the intelligent acceleration engine based on the first extended instruction for data interaction between the processor and the intelligent acceleration engine.

[0006] According to some embodiments, obtaining a preset first extended instruction includes: setting a data flow interface, a direct configuration interface, and a vector operation acceleration interface between the processor and the intelligent acceleration engine; and obtaining the first extended instruction corresponding to the data flow interface, the direct configuration interface, and the vector operation acceleration interface, respectively.

[0007] According to some embodiments, the first extended instruction corresponding to the data flow interface includes a storage instruction; based on the first extended instruction, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, including: determining the registers contained in the processor; decoding the storage instruction to obtain the storage operation data corresponding to the register; and transferring the storage operation data to the shared cache of the intelligent acceleration engine.

[0008] According to some embodiments, the first extended instruction corresponding to the data flow interface further includes a load instruction. Based on the first extended instruction, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine. The instruction further includes: obtaining data in the shared cache of the intelligent acceleration engine; and decoding the load instruction to transfer the data in the shared cache of the intelligent acceleration engine to the corresponding register in the processor.

[0009] According to some embodiments, the first extended instructions corresponding to the direct configuration interface include level generation instructions and batch write instructions; based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, including: obtaining the graph structure of the data to be calculated to determine the control information of each operation unit in the intelligent acceleration engine and the control information of the data to be calculated; determining the configuration type of each operation unit; according to the configuration type, transmitting the control information of each operation unit in parallel to the corresponding operation unit in the intelligent acceleration engine through level generation instructions and / or batch write instructions; and transmitting the control information of the data to be calculated to the corresponding shared cache in the intelligent acceleration engine through level generation instructions and / or batch write instructions.

[0010] According to some embodiments, the first extended instruction corresponding to the direct configuration interface further includes a pulse generation instruction; based on the first extended instruction, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, and the configuration further includes: based on the data in the shared cache of the intelligent acceleration engine, data transmission between the shared cache of the intelligent acceleration engine and the corresponding computing unit of the intelligent acceleration engine is performed through the pulse generation instruction.

[0011] According to some embodiments, the first extended instructions corresponding to the direct configuration interface also include blocking read instructions, non-blocking read instructions, and batch read instructions; based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, which also includes: determining whether the calculation of the data to be calculated is completed through blocking read instructions; and determining the state of the computing unit of the intelligent acceleration engine through non-blocking read instructions and batch read instructions.

[0012] According to some embodiments, the first extended instructions corresponding to the vector operation acceleration interface include parallel computing instructions; based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, including: configuring the computing unit of the intelligent acceleration engine by directly configuring the first extended instructions corresponding to the interface; obtaining the data to be computed; transmitting the data to be computed to the configured computing unit of the intelligent acceleration engine through parallel computing instructions to obtain the computation result; and transmitting the computation result back to the processor through parallel computing instructions.

[0013] According to one aspect of the present invention, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs; wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the method as described above.

[0014] According to one aspect of the present invention, a computer-readable storage medium is provided having a computer program or instructions stored thereon, which, when executed by a processor, implement the method as described above.

[0015] According to embodiments of the present invention, the processor's system bus can be bypassed, and data interaction between the processor and the intelligent acceleration engine can be performed quickly through extended instructions, reducing the idle time between the processor and the intelligent acceleration engine and significantly improving the computing speed and efficiency of heterogeneous intelligent chips.

[0016] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit the invention. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention.

[0018] Figure 1 A schematic diagram of the structure of a heterogeneous smart chip according to an exemplary embodiment of the present invention is shown.

[0019] Figure 2 A flowchart illustrating a data interaction method based on extended instructions applied to heterogeneous smart chips according to an exemplary embodiment of the present invention is shown.

[0020] Figure 3 This diagram illustrates a heterogeneous smart chip interacting with data based on data flow instructions in extended instructions, according to an exemplary embodiment of the present invention.

[0021] Figure 4 This diagram illustrates a heterogeneous smart chip interacting with data based on direct configuration instructions in extended instructions, according to an exemplary embodiment of the present invention.

[0022] Figure 5 This diagram illustrates a heterogeneous smart chip interacting with data based on parallel computing instructions in an extended instruction set, according to an exemplary embodiment of the present invention.

[0023] Figure 6 A block diagram of an electronic device according to an example embodiment of this application is shown. Detailed Implementation

[0024] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that the invention will be thorough and complete, and the concept of the exemplary embodiments will be fully conveyed to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0025] The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a full understanding of embodiments of the invention. However, those skilled in the art will recognize that the invention can be practiced without one or more of these specific details, or other methods, components, materials, apparatus, or operations may be employed. In these cases, well-known structures, methods, apparatuses, implementations, materials, or operations will not be shown or described in detail.

[0026] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0027] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0028] This invention provides a data interaction method, device, and storage medium based on extended instructions, which can quickly perform data interaction between the processor and the intelligent acceleration engine through extended instructions.

[0029] A data interaction method, apparatus, and storage medium based on extended instructions according to an embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0030] Figure 1 A schematic diagram of the structure of a heterogeneous smart chip according to an exemplary embodiment of the present invention is shown.

[0031] like Figure 1 As shown, the heterogeneous smart chip 10 includes a processor 100, a smart acceleration engine 200, and a fast interface 300.

[0032] The processor 100 includes a diffusion instruction processing unit 110.

[0033] The diffusion instruction processing unit 110 is used to obtain a preset first extension instruction and configure the processor 100 and the intelligent acceleration engine 200 based on the first extension instruction to perform data interaction between the processor 100 and the intelligent acceleration engine 200.

[0034] According to some embodiments, processor 100 uses the RISC-V instruction set. The first extended instruction is a custom extended instruction obtained by custom extension of the RISC-V instruction set.

[0035] The intelligent acceleration engine 200 includes a data distribution and collection unit 210.

[0036] The data distribution and collection unit 210 is used to parse the data transmitted by the processor 100 and allocate it to the corresponding computing units in the intelligent acceleration engine 200 for calculation. After the calculation is completed, the data distribution and collection unit 210 feeds back the calculation result to the processor 100.

[0037] A fast interface 300 is provided between the processor 100 and the intelligent acceleration engine 200 so that the processor 100 and the intelligent acceleration engine 200 can be directly connected and transmit data through the fast interface 300.

[0038] According to some embodiments, the fast interface 300 includes a data stream interface 310, a direct configuration interface 320, and a vector operation acceleration interface 330.

[0039] The data flow interface 310 can be used to access the memory space in the intelligent acceleration engine 200, and to quickly load or store data between the registers of the processor 100 and the shared cache of the intelligent acceleration engine 200 through the first extended instruction.

[0040] The direct configuration interface 320 can be used to perform operations such as quick startup and status monitoring of the intelligent acceleration engine 200, and to realize data transmission control and calculation process monitoring of the intelligent acceleration engine 200 by the processor 100 through the first extended instruction.

[0041] The vector operation acceleration interface 330 can be used to directly call the operation unit in the intelligent acceleration engine 200, realize the rapid completion of parallel computation of complex operators through the first extended instruction, and return the calculation result to the processor 100.

[0042] Figure 2 A flowchart illustrating a data interaction method based on extended instructions applied to heterogeneous smart chips according to an exemplary embodiment of the present invention is shown.

[0043] like Figure 2As shown, in step S100, a preset first extended instruction is obtained.

[0044] A fast interface is set up between the processor and the intelligent acceleration engine so that the processor can directly transfer data with the intelligent acceleration engine.

[0045] According to some embodiments, the fast interface includes a data stream interface, a direct configuration interface, and a vector operation acceleration interface.

[0046] The first extended instructions corresponding to the data stream interface, the direct configuration interface, and the vector operation acceleration interface, respectively.

[0047] According to some embodiments, the first extended instruction is a custom extended instruction obtained by custom extension based on the RISC-V instruction set, using the custom -3 reserved instruction code in the RISC-V instruction set, that is, the opcode of the first extended instruction is always "111_1011".

[0048] According to some embodiments, the first extended instruction follows the R-type instruction format to minimize the impact on processor instruction decoding. The R-type instruction format is defined as shown in Table 1. Table 1. Instruction Format Table for R-type

[0049] According to some embodiments, the first extended instructions are divided into data flow instructions corresponding to the data flow interface, direct configuration instructions corresponding to the direct configuration interface, and parallel computing instructions corresponding to the vector operation acceleration interface, based on the type of the fast interface.

[0050] Data Stream Instruction Section Dataflow instructions include load instructions and store instructions. Each register file type in the processor corresponds to a specific load and store instruction.

[0051] According to some embodiments, the register file types in the processor include XRF, FRF, and VRF, which correspond to scalar registers (also known as general-purpose registers), floating-point arithmetic registers, and vector processing registers, respectively.

[0052] In some embodiments, the load instruction corresponding to the general-purpose registers (i.e., XRF registers) in the processor is the cr_sxload instruction, which is used to load data from the processor's external address space into the XRF registers. The definition of the cr_sxload instruction is shown in Table 2: Table 2 cr_sxload instruction definition table

[0053] The `cr_sxload` instruction is a 32-bit instruction. Its `opcode` field is a fixed value "111_1011", and its `rd` field can be any value (i.e., it is not used, the instruction decoder doesn't care, and subsequent instruction descriptions follow this rule). `funct3` specifies the XRF update method: "101" (data within double quotes represents binary values, and subsequent instruction descriptions follow this rule) means only the lower 16 bits are updated, "110" means only the higher 16 bits are updated, and "111" means all 32 bits are updated. The `rs1` field is the number of the user-defined register `addr_type`, ranging from 0 to 31, corresponding to registers ADDR_TYPE0 to ADDR_TYPE31. `rs2` is the XRF register number, ranging from 1 to 31, corresponding to X1 to X31 XRF registers. `funct7` is a fixed value "100_0010".

[0054] The format of the custom register addr_type is shown in Table 3: Table 3. addr_type register format table

[0055] Here, `addr_type` is a 144-bit custom register, configurable via the Processor Control Status Register (CSR) interface. `addr_base` is the base address of the external address space to be accessed, in bytes. `addr_mask` is the base address mask, obtained by performing a bitwise AND operation with the base address to get the final accessed address. `stride` is the address stride, in 4-byte increments, representing the address interval between every 4 bytes of address space. `len` is the length of the data being accessed, in bytes.

[0056] For example, ADDR_TYPE8 has an addr_base value of hexadecimal 0000_0000_F0F0, an addr_mask value of hexadecimal FFFF_FFFF_FF00, a stride of 2, and a len of 8. This ADDR_TYPE8 means that the base address is 0000_0000_F000. First, it accesses 4 bytes of the space 0000_0000_F000~0000_0000_F003. Since the stride is 2, it means an 8-byte interval. The next 4 bytes of the access address space are 0000_0000_F00C~0000_0000_F00F. A total of 8 bytes of space data are accessed, which meets the data length requirement of len=8.

[0057] For example, the XRF number of the cr_sxload instruction is 5, indicating that data loaded from the outside is stored starting from register X5. The addr_type number is 8, indicating that the external address access mode is retrieved from ADDR_TYPE8. The XRF update mode is "101", meaning only the lower 16 bits of each XRF register are updated. This instruction retrieves 8 bytes of data from the address space defined by register ADDR_TYPE8. Bytes 1-2 are assigned to the lower 16 bits of register X5, bytes 3-4 to register X6, bytes 5-6 to register X7, and bytes 7-8 to register X8.

[0058] In some embodiments, the storage instruction corresponding to the general-purpose registers (i.e., XRF registers) in the processor is the cr_sxstore instruction, which is used to store data from the processor's XRF registers to the external address space. The definition of the cr_sxstore instruction is shown in Table 4: Table 4. Definition of cr_sxstore directive

[0059] The meanings of the instruction fields in cr_sxstore are the same as those in cr_sxload.

[0060] For example, the XRF number of the cr_sxstore instruction is 5, indicating that data to be stored in the external address space is retrieved starting from register X5. The addr_type number is 8, indicating that the external address access mode is retrieved from ADDR_TYPE8. The XRF update mode is "101", meaning only the lower 16 bits of each XRF register are retrieved. This instruction concatenates the lower 16 bits of data from registers X5 to X8 and stores them in the address space defined by register ADDR_TYPE8. The lower 16 bits of register X5 are stored in bytes 1-2 of the external address space, the lower 16 bits of register X6 in bytes 3-4, the lower 16 bits of register X7 in bytes 5-6, and the lower 16 bits of register X8 in bytes 7-8.

[0061] As can be seen, the instructions for cr_sxstore and cr_sxload are basically similar, except that the data read and write directions are reversed.

[0062] In some embodiments, the load instruction corresponding to the floating-point arithmetic register (i.e., the FRF register) in the processor is the cr_sfload instruction, which is used to load data from the processor's external address space into the FRF register. The definition of the cr_sfload instruction is shown in Table 5: Table 5. Definition of cr_sfload Instruction

[0063] The cr_sfload instruction is a 32-bit instruction. The opcode field is a fixed value of "111_1011", the rd field can be any value, funct3 is the FRF update mode, "101" means only the lower 16 bits are updated, "110" means only the higher 16 bits are updated, and "111" means all 32 bits are updated. The rs1 field is the number of the user-defined register addr_type, with a value range of 0~31, corresponding to registers ADDR_TYPE0~ADDR_TYPE31 respectively. The rs2 is the number of the floating-point register FRF, with a value range of 0~31, corresponding to registers 01~F31 respectively. The funct7 is a fixed value of "100_0110".

[0064] The difference between the cr_sfload instruction and the cr_sxload instruction lies in the different processor register objects (FRF register and XRF register).

[0065] According to some embodiments, the storage instruction corresponding to the floating-point arithmetic register (i.e., the FRF register) in the processor is the cr_sfstore instruction, which is used to store data from the processor's FRF register to the external address space. The definition of the cr_sfstore instruction is shown in Table 6: Table 6. Definition of cr_sfstore command

[0066] The cr_sfstore instruction is a 32-bit instruction. The opcode field is a fixed value of "111_1011". The rd field can be any value. funct3 is the FRF fetch mode. "101" means fetching only the lower 16 bits, "110" means fetching only the higher 16 bits, and "111" means fetching all 32 bits. The rs1 field is the number of the user-defined register addr_type, with a value range of 0 to 31, corresponding to registers ADDR_TYPE0 to ADDR_TYPE31 respectively. The rs2 field is the number of the floating-point register FRF, with a value range of 0 to 31, corresponding to registers F0 to F31 respectively. funct7 is a fixed value of "100_0111".

[0067] The meanings of each instruction field in cr_sfstore are the same as those in cr_sfload, but the data read and write directions are reversed.

[0068] The difference between the cr_sfstore instruction and the cr_sxstore instruction lies in the different processor register objects (FRF register and XRF register).

[0069] According to some embodiments, the load instruction corresponding to the vector processing register (i.e., the VRF register) in the processor is the cr_svload instruction, which is used to load data from the processor's external address space into the VRF register, where each VRF register has a bit width of 1024 bits. The definition of the cr_svload instruction is shown in Table 7: Table 7 cr_svload instruction definition table

[0070] The cr_svload instruction is a 32-bit instruction. The opcode field is a fixed value of "111_1011". The rd field can be any value. funct3 is the VRF update mode. "101" means only the lower 512 bits are updated, "110" means only the higher 512 bits are updated, and "111" means all 1024 bits are updated. The rs1 field is the number of the custom register addr_type, with a value range of 0 to 31, corresponding to registers ADDR_TYPE0 to ADDR_TYPE31 respectively. The rs2 is the number of the vector register VRF, with a value range of 0 to 31, corresponding to V0 to V31 VRF registers respectively. funct7 is a fixed value of "100_1000".

[0071] For example, the cr_svload instruction has a VRF number of 2 and an addr_type number of 4. ADDR_TYPE4 defines that the amount of data loaded from external space is 256 bytes. The VRF update mode is "110", which means that only the high 512 bits are updated. In actual operation, bytes 0 to 63 loaded back are assigned to the high 512 bits of V2, bytes 64 to 127 are assigned to the high 512 bits of V3, bytes 128 to 191 are assigned to the high 512 bits of V4, and bytes 192 to 255 are assigned to the high 512 bits of V5.

[0072] According to some embodiments, the storage instruction corresponding to the vector processing register (i.e., the VRF register) in the processor is the cr_svstore instruction, which is used to store data in the VRF register to the external address space of the processor. Each VRF register has a bit width of 1024 bits. The definition of the cr_sxload instruction is shown in Table 8. Table 8. Definition of cr_svstore directives

[0073] The cr_svstore instruction is a 32-bit instruction. The opcode field is a fixed value of "111_1011". The rd field can be any value. funct3 is the VRF retrieval mode: "101" means only the lower 512 bits are retrieved, "110" means only the higher 512 bits are retrieved, and "111" means all 1024 bits are retrieved. The rs1 field is the number of the user-defined register addr_type, with a value range of 0 to 31, corresponding to registers ADDR_TYPE0 to ADDR_TYPE31 respectively. The rs2 field is the number of the vector register VRF, with a value range of 0 to 31, corresponding to V0 to V31 VRF registers respectively. funct7 is a fixed value of "100_1001".

[0074] For example, the cr_svstore instruction with VRF number 2 and addr_type number 4, ADDR_TYPE4, defines storing 256 bytes of data from the VRF to the external address space. The VRF value is "110", which means only the high 512 bits are taken. The actual operation is to store the high 512 bits of VRF2 to bytes 0-63 of the external address space, store the high 512 bits of VRF3 to bytes 64-127 of the external address space, store the high 512 bits of VRF4 to bytes 128-191 of the external address space, and store the high 512 bits of VRF5 to bytes 192-255 of the external address space.

[0075] After the aforementioned data stream instructions are decoded into extended instructions, they interact with the intelligent acceleration engine through the data stream interface to transmit data. This eliminates the need for the processor's level 1, level 2, and level 3 caches, significantly improving the loading / storage speed.

[0076] Direct configuration instruction section According to some embodiments, the direct configuration instruction includes the pulse generation instruction cr_spls, which generates a clock cycle of pulse signals (up to 32) at multiple output interfaces based on the value of the general-purpose register XRF, to control the computation start-up, state clearing, etc. of relevant modules within the intelligent acceleration engine. The definition of the cr_spls instruction is shown in Table 9: Table 9. Definition of cr_spls command

[0077] Among them, the cr_spls instruction is a 32-bit instruction, the opcode field is a fixed value "111_1011", funct3 is the pulse type type, type 000 generates a low-level pulse, type 111 generates a high-level pulse, the rd and rs1 fields can be arbitrary values, rs2 is the general-purpose XRF register number, which can be 1~31, corresponding to registers X1~X31. The value of this general-purpose register represents the mask, which is effective for ports with 32 bits set to 1, and ineffective for ports with 32 bits set to 0. funct7 is a fixed value "100_1010".

[0078] For example, when the XRF number of the cr_spls instruction is 5, and the value in X5 is 32'h0000_0f0f, and the type is 111, then a high-level pulse signal of one clock cycle will be generated simultaneously in ports 0 to 3 and ports 8 to 11 of the 32 pulse ports.

[0079] According to some embodiments, the direct configuration instruction includes the level generation instruction cr_slvl, which generates corresponding level signals (up to 32) at multiple output interfaces based on the value of the general-purpose register XRF, to control the clock enable, power enable, path selection, etc. of relevant modules in the intelligent acceleration engine. The definition of the cr_slvl instruction is shown in Table 10: Table 10 cr_slvl instruction definition table

[0080] Among them, the cr_slvl instruction is a 32-bit instruction, the opcode field is a fixed value "111_1011", funct3 is the level type, type 000 generates a low level, type 010 indicates that the original interface level is inverted, and type 111 generates a high level. The rd and rs1 fields can be arbitrary values. rs2 is the general-purpose XRF register number, which can be 1~31, corresponding to registers X1~X31. The value of this general-purpose register represents the mask, which is effective for ports with 32 bits set to 1 and ineffective for ports with 32 bits set to 0. funct7 is a fixed value "100_1011".

[0081] For example, when the XRF number of the cr_slvl instruction is 5, and the value in X5 is 32'h0000_000f, and the type is 111, then a high-level output will be generated at ports 0 to 3 out of the 32 pulse ports.

[0082] For example, when the XRF number of the cr_slvl instruction is 5, and the value in X5 is 32'h0000_00f0, and the type is 010, then the original level of ports 4 to 7 out of the 32 pulse ports will be inverted.

[0083] According to some embodiments, the direct configuration instruction includes the blocking read instruction cr_spoll, which is used to block reads of intelligent acceleration engine signals (up to 32) from multiple ports until the corresponding level is read. The processor then continues executing subsequent instructions; otherwise, it waits. This is more efficient than traditional processor polling reads. The definition of the cr_spoll instruction is shown in Table 11. Table 11 cr_spoll command definition table

[0084] Among them, the cr_spoll instruction is a 32-bit instruction, the opcode field is a fixed value "111_1011", funct3 is the pulse type type, type 000 indicates that a low level is required, type 111 indicates that a high level is required, the rd and rs1 fields can be arbitrary values, rs2 is the general-purpose XRF register number, which can be 1~31, corresponding to registers X1~X31. The value of this general-purpose register represents the mask, which is effective for ports with 32 bits set to 1, and ineffective for ports with 32 bits set to 0. funct7 is a fixed value "100_1110".

[0085] For example, if the XRF number of the cr_spoll instruction is 0, and the value of the X0 register is hexadecimal 0007, and the type is 111, it means that when this instruction is executed, the values ​​of the 0th to 2nd input ports will be read until the level of these 3 ports is all high before continuing to execute subsequent instructions; otherwise, it will remain at this instruction.

[0086] According to some embodiments, the direct configuration instruction includes the non-blocking read instruction cr_srd, used to perform non-blocking reads of intelligent acceleration engine signals (up to 32) from multiple ports. The read-back data is placed in a general-purpose register to monitor the operating status of the intelligent acceleration engine. The definition of the cr_srd instruction is shown in Table 12: Table 12 cr_srd instruction definition table

[0087] Among them, the cr_srd instruction is a 32-bit instruction, the opcode field is a fixed value "111_1011", the funct3, rd, and rs1 fields can be any value, rs2 is the XRF register number, which can be 1~31, corresponding to 1~31 general-purpose registers, and funct7 is a fixed value "100_1111".

[0088] For example, the XRF number of the cr_srd instruction is 5. When this instruction is executed, the level values ​​of the 32 input ports are stored in the X5 register.

[0089] According to some embodiments, the direct configuration instruction includes the bulk read instruction cr_sbrd, used to bulk read the Smart Acceleration Engine signals of the 256-bit input interface into multiple (up to 8) general-purpose registers starting with XRF numbers, in order to read the status or configuration registers of the Smart Acceleration Engine back into the processor. The definition of the cr_sbrd instruction is shown in Table 13: Table 13 cr_sbrd command definition table

[0090] Among them, the cr_sbrd instruction is a 32-bit instruction, the opcode field is a fixed value "111_1011", funct3 is size, which represents the size of the data read in 32 bits, the rd and rs1 fields can be any value, rs2 is the XRF register number, which can be 1~31, corresponding to 1~31 general-purpose registers, and funct7 is a fixed value "101_0000".

[0091] For example, if the cr_sbrd instruction has an XRF number of 5 and a size of 2, then the lower 64 bits of data from the 256-bit input interface will be read back and placed into the X5 and X6 registers.

[0092] According to some embodiments, the direct configuration instruction includes the batch write instruction cr_sbwr, used to batch configure the intelligent acceleration engine signals of the 256-bit output interface. The configuration values ​​come from multiple (up to 8) general-purpose registers starting with XRF numbers, batch configuring data from the processor's registers into the control registers of the intelligent acceleration engine, significantly accelerating the configuration process. The definition of the cr_sbwr instruction is shown in Table 14: Table 14 cr_sbwr instruction definition table

[0093] Among them, the cr_sbwr instruction is a 32-bit instruction, the opcode field is a fixed value "111_1011", funct3 is size, which represents the size of the configured data in 32 bits, the rd and rs1 fields can be any value, rs2 is the XRF register number, which can be 1~31, corresponding to 1~31 general-purpose registers, and funct7 is a fixed value "101_0001".

[0094] For example, if the XRF number of the cr_sbwr instruction is 7 and the size is 2, then the values ​​in the X7 and X8 registers are used to configure the lower 64 bits of the 256-bit output interface.

[0095] After the extended instruction is decoded, the aforementioned direct configuration instructions interact with the intelligent acceleration engine through the direct configuration interface to transmit data. This eliminates the need for the processor's level 1, level 2, and level 3 caches, significantly improving the control speed.

[0096] Parallel computing instruction section According to some embodiments, the parallel computing instruction includes the cr_svfunc0 instruction, which outputs the value of a 1024-bit wide vector processing register in the processor to the vector operation acceleration interface. After the intelligent acceleration engine completes the monocular calculation according to the VRF update method, it writes the calculation result back to another 1024-bit wide vector processing register (VRF register). The definition of the cr_svfunc0 instruction is shown in Table 15: Table 15 defines the cr_svfunc0 instruction.

[0097] In this instruction, cr_svfunc0 is a 32-bit instruction with a fixed opcode field value of "111_1011". funct3 specifies the VRF update mode: "000" indicates that 1024 bits of data are divided into 32 32-bit FP32 data groups, and the result of all 32 data groups is calculated; "001" indicates that 1024 bits of data are divided into 32 32-bit FP32 data groups, and the result of 16 of these data groups is calculated; and "100" indicates that 1024 bits of data are divided into 32 16-bit FP1 data groups. The program calculates the result of all 32 data points. "101" indicates that the 1024-bit data is divided into 32 16-bit FP32 data points, and the result of 16 of these data points is calculated. "rd" indicates the destination VRF register number, which can be 0~31, corresponding to different VRF registers used to store the calculation result. The "rs1" field can be any value. "rs2" is the VRF register number, which can be 1~31, corresponding to vector processing registers 0~31, used to store the data to be calculated. "funct7" is a fixed value "110_0000".

[0098] For example, the source VRF number of the cr_svfunc0 instruction is 0, the destination VRF number is 1, and the VRF update mode is "101". When this instruction is executed, the 1024 bits of data in the V0 register are divided into 32 equal parts. Only the lower 16 bits of each 32-bit data are valid. After being output to the external intelligent acceleration engine, unary operations (such as exp, ln, reciprocal, etc.) are performed. The calculation results are stored in the 32 areas of V1 respectively. The lower 16 bits of data in each area are valid, and the higher bits are padded with 0.

[0099] Similarly, the parallel computing instruction cr_svfunc0 can be extended as shown in Table 16-18: Table 16 Extended cr_svfunc1 instruction definition table

[0100] Table 17. Definition of the extended cr_svfunc2 instruction

[0101] Table 18. Definition of the extended cr_svfunc3 instruction

[0102] The parallel computing instructions described above retrieve the data to be computed from the vector processing register in the processor, transmit it to the intelligent acceleration engine through the vector operation acceleration interface, complete the computation, and then return it to another vector processing register in the processor.

[0103] Parallel computing instructions can perform unary operations, but are not limited to specific computational functions. They can be used to calculate exponents such as y=e^x, reciprocals such as y=1 / x, and logarithms such as y=ln(x). The instructions can be extended, and the processor and intelligent acceleration engine can assign a computational function to each instruction according to a pre-defined agreement.

[0104] The aforementioned parallel computing instructions can fully utilize the complex computing resources in the intelligent acceleration engine to quickly complete the parallel computing of complex operators, thereby accelerating the processor's related computing speed.

[0105] In step S200, the processor and the intelligent acceleration engine are configured based on the first extended instructions for data interaction between the processor and the intelligent acceleration engine.

[0106] Obtain the storage and loading instructions corresponding to the data stream interface in the first extended instruction.

[0107] According to some embodiments, in Large Language Model (LLM) inference, some computations need to be performed within the processor, such as the process of tokenizing strings that segment text into discrete token units. After the processor finishes processing, the data is transferred to the shared cache of the intelligent acceleration engine. The computing units in the intelligent acceleration engine retrieve the data from the shared cache and perform the remaining computations (such as prefilling and decoding).

[0108] According to some embodiments, during convolutional neural network (CNN) inference, image pre- and post-processing is typically performed within the processor, convolution operations are performed in the intelligent acceleration engine, and data also needs to be moved between the two.

[0109] In the traditional data interaction method, the processor first synchronizes data with the system memory, and then the system memory synchronizes with the shared cache in the intelligent acceleration engine. This method is inefficient, slow, and may result in situations where the computing units are waiting for data.

[0110] According to embodiments of the present invention, data can be directly and quickly moved between the registers (XRF / FRF / VRF) in the processor and the shared cache in the intelligent acceleration engine in the form of a data stream through the storage instructions and load instructions corresponding to the data stream interface, which greatly improves the data transmission efficiency.

[0111] like Figure 3 As shown, during a storage operation, the spread instruction processing unit 110 in the processor 100 decodes the storage instruction and converts the storage operation data in the register 120 into data stream data. This data is then sent to the data distribution and collection unit 210 in the intelligent acceleration engine 200 via the data stream interface 310. The data distribution and collection unit 210 parses the data stream interface data and distributes the data to multiple shared caches 230 according to the address information.

[0112] like Figure 3 As shown, during the loading operation, the diffusion instruction processing unit 110 in processor 100 decodes the loading instruction. The data distribution and collection unit 210 in intelligent acceleration engine 200 collects relevant data in shared cache 230 and transmits it to the diffusion instruction processing unit 110 in processor 100 through data flow interface 310. The diffusion instruction processing unit 110 distributes the data to the corresponding registers 120.

[0113] like Figure 3 As shown, the processor 100 and the intelligent acceleration engine 200 form a bidirectional data path through the data flow interface 310. Each data direction includes information such as data validity flag, sending address, sending data size, and whether the data and receiving end are ready.

[0114] Retrieve pulse generation instructions, level generation instructions, blocking read instructions, non-blocking read instructions, batch read instructions, and batch write instructions from the direct configuration instructions.

[0115] According to some embodiments, the processor determines the operations (such as convolution, matrix multiplication size, data path selection, etc.) that each computing unit in the intelligent acceleration engine needs to perform based on the graph structure of the data to be computed, and then configures the control information of the computing units into the control registers (CSRs) of each computing unit.

[0116] The data to be computed includes feature maps, weight parameters, etc., and is stored in the external storage (such as DDR SDRAM) of the heterogeneous intelligent chip. Based on the graph structure of the data to be computed, the processor determines the amount of data, data address, and data address change method of the data to be read from the external storage, which serves as the control information for the data to be computed. The control information for the data to be computed is then configured into the Direct Memory Access Controller (DMA) of the shared cache of the intelligent acceleration engine.

[0117] Furthermore, the direct memory access controller with shared cache is activated to transmit data. After receiving the data, the arithmetic unit performs calculations based on the data stream instructions corresponding to the data stream interface. During the calculation process, the calculation process of the data to be calculated can be monitored by polling the status of the arithmetic unit.

[0118] Traditional data interaction methods configure the control registers in the intelligent acceleration engine one by one via the processor's system bus, but this configuration method has significant latency. In dataflow architecture chips, the combination of various computing modules is flexible and varied, thus requiring the configuration of more control registers, making the configuration latency issue particularly prominent and consequently affecting chip utilization efficiency.

[0119] According to embodiments of the present invention, the intelligent acceleration engine can be quickly configured by directly configuring the pulse generation instructions, level generation instructions, blocking read instructions, non-blocking read instructions, batch read instructions, and batch write instructions corresponding to the interface.

[0120] like Figure 4 As shown, the processor 100 obtains the graph structure of the data to be calculated to determine the control information of each computing unit 220 in the intelligent acceleration engine 200 and the control information of the data to be calculated, and temporarily stores the above control information in the general-purpose register 121 (XRF).

[0121] The registers for clock enable (single-bit level signal), data path selection (multi-bit signal), and operating mode (multi-bit signal) waiting to be configured for each arithmetic unit 220 in the intelligent acceleration engine 200 are determined to determine the configuration type of each arithmetic unit 220.

[0122] Since the data in the arithmetic unit 220 of the intelligent acceleration engine 200 may be large, such as 4, 16, 32, or 64, the number of registers in the arithmetic unit 220 that need to be configured is relatively large. Single-bit signals of the registers of the arithmetic unit 220 can be configured using level generation instructions, and multi-bit signals of the registers of the arithmetic unit 220 can be configured in batches using batch write instructions.

[0123] like Figure 4As shown, the processor 100, through the instruction processing unit 110, outputs the control information of the arithmetic units 220, which have been configured by level generation instructions and / or batch write instructions and are temporarily stored in the general-purpose register 121, to the intelligent acceleration engine 200 in parallel via the direct configuration interface 320 in a preset format. After receiving the parallel configuration data, the data distribution and collection unit 210 in the intelligent acceleration engine 200 distributes it to the control registers (CSRs) of each arithmetic unit 220.

[0124] Similarly, the processor 100 can also transmit control information of the data to be computed, configured by level generation instructions and / or batch write instructions, in a preset format, through the direct configuration interface 320 to the direct memory access controller (DMA) of the shared cache 230 of the intelligent acceleration engine 200.

[0125] Furthermore, such as Figure 4 As shown, a pulse generation instruction activates the direct memory access controller (DMA) of the shared cache 230, enabling the shared cache 230 to transfer data to the corresponding arithmetic unit 220. Upon receiving control information regarding the data to be calculated, the arithmetic unit 220 performs data calculations based on the first extended instruction corresponding to the data stream interface.

[0126] During the calculation process, the processor 100 can obtain the calculation completion flag through the blocking read instruction to determine whether the calculation of the data to be calculated is complete.

[0127] After the calculation is completed, the processor 100 can read the status registers of each arithmetic unit 220 in the intelligent acceleration engine 200 through non-blocking read instructions and batch read instructions to determine the status of the arithmetic unit 220.

[0128] Obtain the parallel computing instructions corresponding to the vector operation acceleration interface.

[0129] According to some embodiments, the processor can invoke computing resources in the intelligent acceleration engine through the first extended instruction corresponding to the vector operation acceleration interface to complete calculations that the processor does not originally support.

[0130] For example, the computational units within the intelligent acceleration engine can perform complex operations such as calculating the exponent y=e^x, the reciprocal y=1 / x, the logarithm y=ln(x), and the square root, some of which are not supported by the processor. In this case, the processor can directly transfer the data to be calculated to each computational unit in the intelligent acceleration engine through parallel computing instructions, complete the calculation, and then return the calculation result to the processor's vector processing register (VRF).

[0131] like Figure 5As shown, after configuring the computing unit 220 in the intelligent acceleration engine 200 via direct configuration instructions, the processor 100 transmits the acquired data to be computed to the computing unit 220 in the intelligent acceleration engine 200 through the vector computation acceleration interface 330 based on parallel computing instructions for data computation. The data transmitted through the vector computation acceleration interface 330 includes a data validity flag and the numerical value of the data to be computed.

[0132] After the calculation is completed, based on parallel computing instructions, the calculation result is returned to the general-purpose register (VRF) of the processor 100 through the vector operation acceleration interface 330. The data returned through the vector operation acceleration interface 330 includes a data validity flag and the numerical value of the calculation result.

[0133] Unlike traditional data interaction methods, in the embodiments of this application, some computing units in the intelligent acceleration engine can be regarded as vector computing units in the processor. Each computing unit has a fixed computing delay, and the processor compilation software can allocate instructions according to the fixed delay information to achieve the highest possible instruction execution efficiency.

[0134] In the embodiments of this application, the computational functions of each parallel computing instruction are not specified, nor are the dedicated data interfaces of each parallel computing instruction bound to the computing units in the intelligent acceleration engine. The specific functions implemented by the parallel computing instructions and the selection of computing units are flexibly determined through the compiler software and the register configuration values ​​in the intelligent acceleration engine. Sufficient parallel computing instruction interfaces can be reserved during processor design, and some interfaces can be redefined for specific functions in future functional expansions, requiring only modifications to the corresponding compiler software.

[0135] Figure 6 A block diagram of an electronic device according to an example embodiment of this application is shown.

[0136] like Figure 6 As shown, the electronic device 600 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0137] like Figure 6 As shown, the electronic device 600 is manifested in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc. The storage unit stores program code, which can be executed by the processing unit 610, causing the processing unit 610 to perform the methods described in this specification according to the various exemplary embodiments of this application. For example, the processing unit 610 can perform, for example... Figure 2 The method shown.

[0138] Storage unit 620 may include readable media in the form of volatile storage units, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include read-only memory (ROM) 6203.

[0139] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0140] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0141] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0142] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. The technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this application.

[0143] Software products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0144] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0145] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0146] The aforementioned computer-readable medium carries one or more programs, which, when executed by a device, cause the computer-readable medium to perform the aforementioned functions.

[0147] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0148] The embodiments of the present invention have been described in detail above. These descriptions are solely for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, any changes or modifications made by those skilled in the art based on the ideas of the present invention, its specific implementation methods, and its application scope, are all within the scope of protection of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data interaction method based on extended instructions, applied to a heterogeneous intelligent chip, wherein the heterogeneous intelligent chip includes a processor and an intelligent acceleration engine, characterized in that, include: Retrieve the preset first extended instruction; Based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine.

2. The method according to claim 1, characterized in that, Retrieve the preset first extended instruction, including: A data flow interface, a direct configuration interface, and a vector operation acceleration interface are provided between the processor and the intelligent acceleration engine; Obtain the first extended instructions corresponding to the data stream interface, the direct configuration interface, and the vector operation acceleration interface, respectively.

3. The method according to claim 2, characterized in that, The first extended instruction corresponding to the data stream interface includes a storage instruction; Based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, including: Determine the registers contained in the processor; The storage instruction is decoded to obtain the storage operation data corresponding to the register; The storage operation data is transferred to the shared cache of the intelligent acceleration engine.

4. The method according to claim 3, characterized in that, The first extended instruction corresponding to the data stream interface also includes a load instruction. Based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, further comprising: Retrieve data from the shared cache of the intelligent acceleration engine; The loading instruction is decoded to transfer data from the shared cache of the intelligent acceleration engine to the corresponding registers in the processor.

5. The method according to claim 2, characterized in that, The first extended instructions corresponding to the direct configuration interface include level generation instructions and batch write instructions; Based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, including: Obtain the graph structure of the data to be computed to determine the control information of each computing unit in the intelligent acceleration engine and the control information of the data to be computed. Determine the configuration type of each computing unit; According to the configuration type, the control information of each computing unit is transmitted in parallel to the corresponding computing unit in the intelligent acceleration engine via the level generation instruction and / or the batch write instruction; and The control information of the data to be calculated is transmitted to the corresponding shared cache in the intelligent acceleration engine through the level generation instruction and / or the batch write instruction.

6. The method according to claim 5, characterized in that, The first extended instruction corresponding to the direct configuration interface also includes a pulse generation instruction; Based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, further comprising: Based on the data in the shared cache of the intelligent acceleration engine, data transmission between the shared cache of the intelligent acceleration engine and the corresponding computing unit of the intelligent acceleration engine is performed through the pulse generation instruction.

7. The method according to claim 5, characterized in that, The first extended instructions corresponding to the direct configuration interface also include blocking read instructions, non-blocking read instructions, and batch read instructions; Based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, further comprising: The completion of the calculation of the data to be calculated is determined by the blocking read instruction; and The state of the computing unit of the intelligent acceleration engine is determined by the non-blocking read instruction and the batch read instruction.

8. The method according to claim 2, characterized in that, The first extended instructions corresponding to the vector operation acceleration interface include parallel computing instructions; Based on the first extended instructions, the processor and the intelligent acceleration engine are configured for data interaction between the processor and the intelligent acceleration engine, including: The computing unit of the intelligent acceleration engine is configured through the first extended instruction corresponding to the direct configuration interface; Obtain the data to be calculated; The parallel computing instructions transmit the data to be computed to the processing unit of the configured intelligent acceleration engine to obtain the computation result; and The computation result is sent back to the processor via the parallel computation instructions.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the method as described in any one of claims 1-8.

Images