Artificial intelligence system-on-chip and artificial intelligence computing system

Abstract

An artificial intelligence system-on-a-chip (AI system-on-a-chip) is disclosed, suitable for high-performance, low-power AI model processing in heterogeneous computing platforms. The AI ​​system-on-a-chip includes a RISC-V processor, an ASIP (Automatic System-on-IP), an NPU (Natural Processing Unit), a memory module, and an interconnect module. The ASIP has a task dispatcher and a system control unit, and dynamically schedules computing tasks and data flows based on the real-time execution status of the NPU and RISC-V processor through multiplexing scheduling control logic, coordinating data transmission and control between modules. The interconnect module supports handshaking mechanisms, dynamic data channel switching, and priority arbitration to enhance data flow efficiency. Furthermore, this invention also discloses an AI computing system, including the aforementioned AI system-on-a-chip and an AI model compiler. The AI ​​model compiler compiles external model code into NPU-executable instruction code and deploys it to the memory module. Through hardware and software co-design, this invention can dynamically adjust computing strategies according to model characteristics and system resources, improving overall computing performance and flexibility, and is suitable for diverse application scenarios such as AI edge computing, medical image analysis, and industrial smart inspection.

Claims

1. An artificial intelligence system chip, comprising: A RISC-V processor for handling non-AI tasks; A dedicated instruction set processor (ASIP) for task assignment, I / O control, and data flow management; a neural network processor (NPU) for performing model computations for artificial intelligence models; and a memory module. The system also includes an interconnect module that connects the RISC-V processor, ASIP, NPU, and memory module. The interconnect module has a handshake mechanism to support dynamic switching of data channels and priority arbitration control of task data flow. The ASIP includes a task dispatcher and a system control unit. The task dispatcher controls the execution order of each task corresponding to the RISC-V processor, ASIP, or NPU. The system control unit includes multiplexing scheduling control logic and dynamically schedules task assignment based on the real-time execution status and resource usage of the RISC-V processor and NPU. The interconnect module performs data transmission to achieve coordinated operation of data flow and control flow.

2. The artificial intelligence system chip as described in claim 1 further includes an I / O interface for receiving a task assignment table generated by an artificial intelligence model compiler, storing the task assignment table in a memory module for the ASIP's task dispatcher to read and parse, and thereby control the RISC-V processor, ASIP or NPU corresponding to each task and their execution order.

3. The artificial intelligence system chip as described in claim 2, wherein, The NPU supports receiving fixed-point format model control codes output by the AI ​​model compiler. The fixed-point format uses a write strobe (WSTRB) format to mark valid data bytes, thereby reducing the model data size and improving model computation efficiency.

4. The artificial intelligence system chip as described in claim 1, wherein, The interconnect module further supports transmission optimization configurations based on different data types, including: (a) weight data, which are reusable parameter values ​​in the artificial intelligence model. The interconnect module is configured with a reuse buffer to reduce the number of reads from external memory; (b) feature map data, which are intermediate computation data passed between layers of the neural network model. The interconnect module is configured with a ping-pong buffer to dynamically switch buffers between read and write operations to avoid data transmission congestion; and (c) control command data, which are task control instructions generated by the ASIP. These instructions are transmitted through the interconnect module to the corresponding RISC-V processor or NPU, and the NPU or RISC-V processor loads the required control instruction code from the corresponding memory area of ​​the memory module to execute the corresponding task.

5. The artificial intelligence system chip as described in claim 1, wherein, The memory module has multiple memory regions, including: weight memory, feature map memory, RISC-V data memory, ASIP instruction memory, ASIP data memory, and RISC-V instruction memory.

6. The artificial intelligence system chip as described in claims 1, 2, 3, 4, or 5, wherein, The task dispatcher determines the suitability of task execution. If it determines that the NPU does not support the task or is inefficient, it will reassign the task to the RISC-V processor or the ASIP for execution.

7. The artificial intelligence system chip as described in claim 1 further includes an I / O interface, through which the ASIP can receive control instructions provided by a dedicated instruction set compiler, wherein the control instructions are generated according to different application requirements, and the control instructions are transmitted to the memory module for storage through the interconnect module.

8. The artificial intelligence system chip as claimed in claim 1 further includes an IO interface controlled by the ASIP for exchanging data with external devices and performing data transfer and classification processing through the interconnect module.

9. The artificial intelligence system chip as described in claim 1, wherein, The interconnect module includes: a bridge having a master device interface for connecting at least one master device and a slave device interface for connecting at least one slave device, responsible for coordinating the handshake mechanism and data transmission process between the master and slave devices; an address decoding module connected to the slave device interface, which parses access requests from the master device to determine the target address of the access request and then determines the slave device corresponding to the target address; and a master switching circuit and a slave switching circuit connected to the master device interface and the slave device interface, respectively. The master switching circuit coordinates the access requests of each master device to the memory module when multiple master devices initiate requests, and the slave switching circuit receives data requests from the master switching circuit and the bridge, and the address decoding module parses the target address of the data request to transmit the data to the slave device specified by the target address for storage.

10. The artificial intelligence system chip as described in claim 9, wherein, The master device is a RISC-V processor, ASIP, NPU, or IO interface, and the slave device is a memory module with multiple memory regions, each of which corresponds to a dedicated target address range. These multiple memory regions include: Weight MEM, Feature Map MEM, RISC-V Data Memory, ASIP Instruction Memory, ASIP Data Memory, and RISC-V Instruction Memory.

11. The artificial intelligence system chip as described in claim 10, wherein, The I / O interface is a Serial Peripheral Interface (SPI). The bridge and address decoding module determine the target address of the slave device based on the address signal (WADDR) to be transmitted by the SPI, and complete the data transmission and access.

12. The artificial intelligence system chip as described in claim 9, 10, or 11, wherein, The bridge uses a handshake mechanism to ensure the correctness of data transmission between the master device and the slave device. The state transition logic signals during the execution of the handshake mechanism include: data valid signal (WVALID) and data receive ready signal (WREADY), write address (WADDR), write data (WDATA), and a masked write mask signal (WSTRB). The handshake mechanism enables the master device and the slave device to complete a valid data transmission when both the data valid signal (WVALID) and the data receive ready signal (WREADY) are high.

13. The artificial intelligence system chip as described in claim 9, wherein, The bridge implements a handshake mechanism and data transmission control, and confirms the correctness of data transmission based on the handshake mechanism. The handshake mechanism includes at least: a valid signal (VALID) sent by the master device and a ready signal (READY) responded by the slave device.

14. An artificial intelligence computing system, comprising: The artificial intelligence system chip as described in claim 1; And an artificial intelligence model compiler, used to compile external artificial intelligence model program code into instruction code that can be executed by the NPU of the artificial intelligence system chip; wherein, the instruction code can be deployed to the memory module of the artificial intelligence system chip for execution.

15. The artificial intelligence computing system as described in claim 14, wherein, The AI ​​model compiler generates a task assignment table and stores it in the memory module of the AI ​​system chip for the task dispatcher to read and parse, and to control the RISC-V processor, ASIP or NPU corresponding to each task and their execution order.

16. The artificial intelligence computing system as described in claim 14, wherein, The AI ​​model compiler includes a quantization module to convert floating-point format weights or model data into fixed-point format and mark the effective data bytes with a write strobe (WSTRB) to reduce the model data size and improve model computation efficiency. The converted fixed-point format weights or model data can be received and executed by the NPU of the AI ​​system chip and deployed in the memory module for AI model computation.

17. The artificial intelligence computing system as described in claim 14, further comprising: A dedicated instruction set compiler is used to compile the high-level control description that implements the multiplexing scheduling control logic, dynamic scheduling task assignment and data transmission control into low-level control instructions that can be recognized by ASIP, and transmit the control instructions to the memory module for storage through the interconnect module.

Images