A general purpose neural network tensor processor
By adopting the combination of Turing complete instruction set and data stream core in neural network tensor processors, the problem that traditional processors cannot support new algorithms is solved, and a general neural network acceleration with high computing power, low energy consumption and high performance is achieved Computing, with operator scalability, meets the needs of new algorithms.
Patent Information
- Application Number
- CN202210031623.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-12
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2042-01-12
AI Technical Summary
Due to the limitations of finite type operators, traditional neural network tensor processors cannot support new algorithms and operators that cannot be indirectly supported by combining existing operators, resulting in the inability to run new algorithms and cannot meet the needs of the ever-evolving artificial intelligence algorithms. .
Design a general neural network tensor processor, using the instruction stream core and data stream core of the Turing complete instruction set, to achieve mutual access and signal interaction between the cores through the system bus and the peripheral bus, providing high flexibility in computing The sub-scalable capability is that the data flow core is responsible for high computing power calculations, and the instruction flow core is responsible for non-computing tasks, realizing high computing power, low energy consumption, and high performance neural network accelerated computing.
It realizes high computing power, low energy consumption, high performance and flexible neural network accelerated computing, has operator scalability, can support a variety of computing tasks, and meets the needs of new algorithms.
Smart Images

Figure CN114970844B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network tensor processors, and in particular to a general neural network tensor processor. Background Art
[0002] Neural network tensor processors are used to efficiently implement accelerated computing of neural network algorithms. In existing solutions, neural network tensor processors generally only support a limited number of types of calculations, which are generally referred to as operators. Neural network tensor processors achieve accelerated computing of neural network algorithms by supporting a specific number of operators. For information about neural network tensor processors, please refer to the previous patent 1 (the name of the invention is: A neural network multi-core tensor processor, application number: 202011423696.0), or the previous patent 2 (the name of the invention is: A neural network tensor processor, application number: 202011421828.6).
[0003] The current neural network tensor processors are usually based on a limited number of operators. When used specifically, their functions have certain limitations: the neural network tensor processors cannot run on unsupported operators or operators that cannot be indirectly supported by combining existing operators.
[0004] Artificial intelligence technology is in a period of rapid development, and the development speed of artificial intelligence algorithms is accelerating. With the development of technology, new algorithms and new operators continue to emerge. Therefore, traditional neural network tensor processors based on a fixed set of operators can easily fail to support the operation of new algorithms as the algorithms evolve. Summary of the invention
[0005] In order to solve the above problems, the present invention proposes a universal neural network tensor processor with flexible operator scalability to meet the new needs of constantly innovative artificial intelligence algorithms.
[0006] The specific technical solutions proposed by the present invention are as follows:
[0007] A general neural network tensor processor comprises: an instruction stream kernel for realizing the function of executing an instruction set, wherein the instruction set is a Turing complete instruction set; a data stream kernel for configuring and realizing data stream operators; a system bus, which is realized as an AXI system master device interface externally and realizes arbitration between the system master interface of the instruction stream kernel and the data stream kernel internally, and allows the data stream kernel or one of the instruction stream kernels to access an external system at the same time; an instruction stream peripheral bus for realizing switching of the instruction stream kernel's access to peripherals in different address spaces, wherein a part of the address space is mapped to the data stream kernel and a part is mapped to an external AXI bus; and a data stream peripheral bus for arbitrating access requests from the instruction stream kernel or an external AXI device, and allowing one of them to access a register of the data stream kernel at the same time.
[0008] From the perspective of the interface of the functional module, further, the instruction stream kernel includes: a system master interface for obtaining instructions and data from an external memory; a peripheral master interface for actively reading and writing registers of other devices; an external interrupt interface for the external system to trigger the instruction stream kernel to start calculation; and an internal signal interface for realizing signal interaction between the instruction stream kernel and the data stream kernel.
[0009] From the perspective of the interface of the functional module, further, the data flow core includes: a system master interface for obtaining data from an external memory; a peripheral slave interface for being accessed by an instruction flow core or an external master device; and an internal signal interface for realizing signal interaction between the data flow core and the instruction flow core.
[0010] From the perspective of functional modules, further, the instruction stream kernel includes: an instruction pipeline module, which is used to implement the execution function of all instructions; a system bus bit width adaptation module, which is used to convert the data bit width of the instruction pipeline module and make it consistent with the data bit width of the data stream kernel; wherein the data bit width of the data stream kernel is 2 to the Nth power of the data bit width of the instruction pipeline module, wherein N is an integer greater than or equal to 2; an address space configuration module, which is used to implement address space attribute configuration, and the attribute configuration information of the address space comes from the data stream kernel; an interrupt controller module, which is used to receive two types of interrupt trigger signals, one from the data stream kernel and the other from the external system. After receiving the interrupt signal, the interrupt controller will trigger the instruction pipeline to perform different tasks according to different interrupt types; and a data stream kernel control module, which is used to implement interaction with the data stream kernel, and its input and output signals include an interrupt output signal, a synchronous input signal and a synchronous output signal, wherein the interrupt output signal is effectively used to trigger the data stream kernel to start a new task, the synchronous input signal is effectively used to represent the end of the data stream kernel calculation, and the synchronous output signal is effectively used to send the instruction stream kernel calculation end status to the data stream kernel.
[0011] Furthermore, the data bit width of the data stream core is 256 bits, 512 bits or 1024 bits, and the data bit width of the instruction pipeline module is 64 bits or 32 bits.
[0012] Furthermore, the address space attributes of the instruction stream kernel include: shared data address space, which refers to the address space that the instruction stream kernel and the data stream kernel can access simultaneously, and is used to store data shared by the instruction stream kernel and the data stream kernel. It is readable and writable. The data in this address space is stored strictly in accordance with the data bit width alignment of the data stream kernel, and has the highest data read and write performance; temporary data address space, which refers to the address space exclusively used by the instruction stream kernel, and is used to store data used solely by the instruction stream kernel. It is readable and writable. There is no address alignment requirement for the data in this address space, and data can be read and written at the minimum granularity of bytes, with the highest flexibility; and instruction address space, which refers to the address space exclusively used by the instruction stream kernel, and is used to store instructions used solely by the instruction stream kernel. It is read-only, and the instructions in this address space are stored strictly in accordance with the data bit width alignment of the data stream kernel, and have the highest read performance.
[0013] From the perspective of functional modules, further, the data flow kernel includes: a main controller, which is used to provide a control and status interface to the outside so that an external control unit can control the data flow kernel; a data flow reconstruction controller, which is used to configure the data flow calculation module to enable it to implement a certain operator function; a data flow calculation module, which is used to implement data calculation, and its specific function is determined by the data flow reconstruction controller; and an instruction stream kernel controller, which is used to implement the interaction between the data flow kernel and the instruction stream kernel, and its input and output signals include an interrupt input signal, a synchronization input signal, a synchronization output signal, an address control configuration signal and an interrupt output signal, wherein the interrupt input signal is effectively used to trigger the data flow kernel to start a new calculation, the synchronization input signal is effectively used to represent the end of the instruction flow kernel calculation, the synchronization output signal is effectively used to send the status of the end of the data flow kernel calculation to the instruction flow kernel, the address control configuration signal is used to configure the address space attributes of the instruction flow kernel, and the interrupt output signal is effectively used to trigger the instruction flow kernel to start a new task.
[0014] The present invention achieves the following technical effects:
[0015] The general neural network tensor processor of the present invention provides high computing power, low energy consumption and high performance data computing through a built-in data flow kernel with limited computing types and implemented by data flow computing technology, and provides highly flexible data operations (transportation, combination, etc.) through a built-in instruction flow kernel with a Turing complete instruction set, infinitely expandable computing types and implemented by instruction set computing technology. The data flow kernel is mainly responsible for high computing power computing tasks, and the instruction flow kernel is mainly responsible for non-computing tasks. The data flow kernel and the instruction flow kernel complement each other, thereby realizing neural network accelerated computing with operator scalability, high computing power, low energy consumption, high performance and flexible and general purpose. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 is the overall architecture of the processor of the present invention;
[0017] Figure 2 is an example of an instruction stream;
[0018] Figure 3 is an example of a data flow;
[0019] Figure 4 It is the instruction stream kernel architecture of the present invention;
[0020] Figure 5 It is the data flow kernel architecture of the present invention. DETAILED DESCRIPTION
[0021] To further illustrate various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.
[0022] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.
[0023] The present invention provides a general neural network tensor processor overall architecture, such as Figure 1 The general neural network tensor processor proposed in the present invention is composed of five modules: instruction stream core, data stream core, system bus, instruction stream peripheral bus and data stream peripheral bus.
[0024] The instruction stream kernel mainly implements the function of running an instruction set. The instruction refers to a rule for operating data. Different rules can define different instructions. The set of all instructions is called an instruction set. The instruction set can be any instruction set with Turing completeness, such as the RISC-V instruction set. The instruction stream kernel reads the instruction stream and executes each instruction in sequence to implement the computing function defined by the instruction stream. An instruction stream such as Figure 2Although the types of instructions are limited, the combinations of instructions are infinite. Therefore, the computing functions that can be implemented by the instruction stream core are determined by the combination of instructions. The instruction stream core can implement any function defined by the instruction stream.
[0025] The instruction stream core has a system master interface, a peripheral master interface, an external interrupt interface and an internal signal interface. The system master interface is used to obtain instructions and data from the external memory; the peripheral master interface is used to actively read and write registers of other devices; the external interrupt interface is used by the external system to trigger the instruction stream core to start calculation; the internal signal interface is used to realize the signal interaction between the instruction stream core and the data stream core.
[0026] The dataflow kernel mainly implements the function of dataflow operation. The dataflow kernel can realize the calculation of specific functions, which are defined as dataflow operators. The dataflow operator itself has indivisible computing functions, such as convolution, pooling, etc. The set of dataflow operators does not have Turing completeness, and it is impossible to realize any other arbitrary operator through the combination of operators. The dataflow kernel has a limited number of dataflow operator types and can realize limited computing functions. When the operator function of the dataflow kernel is configured, its computing function is determined. At this time, the dataflow kernel reads the data stream and processes each data in sequence to realize data processing of a certain computing function. A dataflow such as Figure 3 shown.
[0027] The data stream core has a system master interface, a peripheral slave interface and an internal signal interface. The system master interface is used to obtain data from the external memory; the peripheral slave interface is used to be accessed by the instruction stream core or an external master device; the internal signal interface is used to realize the signal interaction between the data stream core and the instruction stream core.
[0028] The system bus is implemented as an AXI system master interface externally, mainly used to access external systems (such as DDR memory). The system bus implements arbitration between the system master interfaces of the instruction stream core and the data stream core internally, allowing either the data stream core or the instruction stream core to access the external system at the same time.
[0029] The instruction stream peripheral bus is used to implement the switching of the instruction stream core's access to peripherals in different address spaces. Part of the address space is mapped to the data stream core, and part is mapped to the external AXI bus.
[0030] The data stream peripheral bus is used to arbitrate access requests from the instruction stream core or external AXI devices, and allows one of them to access the registers of the data stream core at the same time.
[0031] The instruction stream core and the data stream core interact with each other through dedicated internal signals, which include three types: synchronization signals, interrupt signals and address space configuration signals.
[0032] The synchronization signal is used to achieve the synchronization of the data flow kernel and the instruction flow kernel. In the actual operation process, the data flow kernel and the instruction flow kernel can run in parallel. When the data flow kernel and the instruction flow kernel execute different parts of the same task, both parties need to wait for each other through the synchronization signal until both calculations are completed and then end at the same time.
[0033] The interrupt signal is used to enable the data stream core to trigger the instruction stream core to start calculation, or the instruction stream core to trigger the data stream core to start calculation.
[0034] The address space configuration signal is used to configure the instruction stream kernel operation address space attributes. The instruction stream kernel operation address space has three attributes, namely shared data address space, temporary data address space and instruction address space.
[0035] The shared data address space refers to the address space that can be accessed by the instruction stream core and the data stream core at the same time. It is used to store data shared by the instruction stream core and the data stream core. It is readable and writable. The data in this address space is stored strictly in a 512-bit aligned manner and has the highest data read and write performance.
[0036] Temporary data address space refers to the address space exclusively used by the instruction stream core. It is used to store data used solely by the instruction stream core. It is readable and writable. There is no requirement for address alignment of the data in this address space. Data can be read and written at the minimum granularity of bytes (8 bits), which has the highest flexibility.
[0037] The instruction address space refers to the address space exclusively used by the instruction stream core. It is used to store instructions used solely by the instruction stream core. It is read-only. The instructions in this address space are stored strictly in a 512-bit aligned manner and have the highest read performance.
[0038] The instruction stream kernel is composed of a system bus width adaptation module, a 64-bit / 32-bit instruction pipeline module, an address space configuration module, an interrupt controller module, and a data stream kernel control module. Figure 4 shown.
[0039] The 64-bit / 32-bit instruction pipeline module implements the execution function of all instructions. The module obtains instructions and data from the system main interface, executes instructions and calculates data to achieve specific functions. The module can also read and write registers of other devices through the peripheral interface. The data bit width of this module is 64 bits or 32 bits.
[0040] The system bus width adaptation module is used to convert the 64-bit or 32-bit data width of the instruction pipeline module and make it consistent with the data width of the data stream core, and the typical value is 512 bits. The data width of the data stream core is set to 2N times the data width of the instruction pipeline module, where N is greater than or equal to 2. In other embodiments, the data width of the data stream core can also be 256 bits or 1024 bits.
[0041] The address space configuration module is used to implement the address space attribute configuration as mentioned above. The attribute configuration information of the address space comes from the data flow kernel. The address space configuration module provides the address space information to the instruction pipeline according to different attribute configurations. The instruction pipeline adopts different read and write strategies when reading and writing data in different address spaces.
[0042] The interrupt controller module is used to receive two types of interrupt trigger signals, one from the data flow core and the other from the external system. After receiving the interrupt signal, the interrupt controller will trigger the instruction pipeline to perform different tasks according to the different interrupt types.
[0043] The data flow kernel control module is mainly used to realize the interaction with the data flow kernel. Among them, the interrupt output signal is valid to trigger the data flow kernel to start a new task. The validity of the synchronous input signal represents the end of the data flow kernel calculation. The validity of the synchronous output signal can send the instruction flow kernel calculation end status to the data flow kernel.
[0044] The data flow kernel consists of a main controller, a data flow reconstruction controller, a data flow calculation module, and an instruction flow kernel controller. Figure 5 shown.
[0045] The role of the main controller is to provide a control and status interface to the outside world so that an external control unit (such as a CPU or instruction stream core connected to the same AXI bus) can control the data stream core, such as configuring the algorithm data address, input data address, calculation enable, etc.
[0046] The main function of the data flow reconstruction controller is to configure the data flow computing module to implement a certain operator function, such as convolution or pooling.
[0047] The main function of the data flow computing module is to realize data calculation, and its specific function is determined by the data flow reconstruction controller.
[0048] The instruction stream kernel controller is mainly used to implement interaction with the instruction stream kernel. Among them, the interrupt input signal is valid to trigger the data stream kernel to start a new calculation. The validity of the synchronization input signal represents the end of the instruction stream kernel calculation. The validity of the synchronization output signal can send the status of the end of the data stream kernel calculation to the instruction stream kernel. The address control configuration signal is used to configure the address space properties of the instruction stream kernel. The validity of the interrupt output signal can trigger the instruction stream kernel to start a new task.
[0049] The general neural network tensor processor of the present invention provides high computing power, low energy consumption and high performance data computing through a built-in data flow kernel with limited computing types and implemented by data flow computing technology, and provides highly flexible data operations (transportation, combination, etc.) through a built-in instruction flow kernel with a Turing complete instruction set, infinitely expandable computing types and implemented by instruction set computing technology. The data flow kernel is mainly responsible for high computing power computing tasks, and the instruction flow kernel is mainly responsible for non-computing tasks. The data flow kernel and the instruction flow kernel complement each other, thereby realizing neural network accelerated computing with operator scalability, high computing power, low energy consumption, high performance and flexible and general purpose.
[0050] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.
Claims
1. A general neural network tensor processor, characterized in that: include: An instruction stream kernel, used to implement the function of executing an instruction set, wherein the instruction set is a Turing complete instruction set; Dataflow kernel, used to configure and implement dataflow operators; The system bus is implemented as an AXI system master interface externally and implements arbitration between the system master interfaces of the instruction stream core and the data stream core internally, allowing either the data stream core or the instruction stream core to access the external system at the same time; An instruction stream peripheral bus, used to implement the switching of the instruction stream core's access to peripherals in different address spaces, where part of the address space is mapped to the data stream core and part is mapped to an external AXI bus; and The data stream peripheral bus is used to arbitrate access requests from the instruction stream core or external AXI devices, and allows one of them to access the registers of the data stream core at the same time; The instruction stream kernel comprises: Instruction pipeline module, used to implement the execution function of all instructions; A system bus bit width adaptation module, used for converting the data bit width of the instruction pipeline module and making it consistent with the data bit width of the data stream core; wherein the data bit width of the data stream core is 2 to the Nth power of the data bit width of the instruction pipeline module, wherein N is an integer greater than or equal to 2; The address space configuration module is used to implement the address space attribute configuration. The address space attribute configuration information comes from the data stream kernel. The interrupt controller module is used to receive two types of interrupt trigger signals, one from the data flow core and the other from the external system. After receiving the interrupt signal, the interrupt controller will trigger the instruction pipeline to perform different tasks according to different interrupt types; and The data flow kernel control module is used to realize the interaction with the data flow kernel. Its input and output signals include an interrupt output signal, a synchronization input signal and a synchronization output signal. Among them, the interrupt output signal is effectively used to trigger the data flow kernel to start a new task, the synchronization input signal is effectively used to represent the end of the data flow kernel calculation, and the synchronization output signal is effectively used to send the instruction flow kernel The status of the end of the calculation is sent to the data flow kernel.
2. The general neural network tensor processor according to claim 1, characterized in that The instruction stream kernel comprises: A system master interface for fetching instructions and data from external memory; A peripheral master interface, used to actively read and write registers of other devices; An external interrupt interface for an external system to trigger the instruction stream core to start calculation; and An internal signal interface used to implement signal interaction between the instruction stream core and the data stream core.
3. The general neural network tensor processor according to claim 1, characterized in that The data flow kernel includes: A system master interface for obtaining data from external memory; A peripheral slave interface for access by the instruction stream core or an external master; and An internal signal interface used to implement signal interaction between the data stream core and the instruction stream core.
4. The general neural network tensor processor according to claim 1, characterized in that The data bit width of the data stream core is 256 bits, 512 bits or 1024 bits, and the data bit width of the instruction pipeline module is 64 bits or 32 bits.
5. The general neural network tensor processor according to claim 1, wherein: The address space properties in which the instruction stream kernel operates include: Shared data address space refers to the address space that can be accessed by the instruction stream core and the data stream core at the same time. It is used to store data shared by the instruction stream core and the data stream core. It is readable and writable. The data in this address space is stored in a strictly aligned manner according to the data bit width of the data stream core. Temporary data address space refers to the address space exclusively used by the instruction stream core, which is used to store data used solely by the instruction stream core. It is readable and writable. There is no requirement for address alignment of the data in this address space. Data can be read and written at the minimum granularity of bytes; and The instruction address space refers to the address space exclusively used by the instruction stream core. It is used to store instructions used solely by the instruction stream core. It is read-only. The instructions in this address space are stored strictly in accordance with the data bit width alignment of the data stream core.
6. The general purpose neural network tensor processor according to claim 1, wherein: The data flow kernel includes: The main controller is used to provide control and status interfaces to the outside world, so that the external control unit can control the data flow core; The data stream reconstruction controller is used to configure the data stream computing module to implement a certain operator function; A data flow calculation module, which is used to realize data calculation, and its specific function is determined by the data flow reconstruction controller; and An instruction stream kernel controller is used to realize the interaction between the data stream kernel and the instruction stream kernel. Its input and output signals include an interrupt input signal, a synchronization input signal, a synchronization output signal, an address control configuration signal and an interrupt output signal. Among them, the interrupt input signal is effectively used to trigger the data stream kernel to start a new calculation, the synchronization input signal is effectively used to represent the end of the instruction stream kernel calculation, the synchronization output signal is effectively used to send the status of the end of the data stream kernel calculation to the instruction stream kernel, the address control configuration signal is used to configure the address space attributes of the instruction stream kernel, and the interrupt output signal is effectively used to trigger the instruction stream kernel to start a new task.
