RISC-V AI Inference Chip Using DMA and Vector Processing

The patent applies universality by designing a multi-functional AI inference chip that can execute both vector instructions (for parallel processing) and scalar instructions (for control logic) on the same RISC-V processor. This allows a single hardware platform to handle diverse AI workloads including convolutional neural networks, recurrent neural networks, and other machine learning models, thereby improving adaptability while maintaining a unified hardware architecture.

The patent segments the instruction processing into two distinct functional units: a vector processing unit for handling vector instructions and a scalar processing unit for handling scalar instructions. This segmentation allows each unit to be optimized for its specific instruction type while both units share the same RISC-V instruction set architecture, resolving the hardware design challenge by providing specialized processing paths within a unified framework.

The patent introduces an intermediary layer of software tools including a compiler that translates high-level AI algorithms into RISC-V assembly code, and a library of pre-optimized instruction sequences for common AI operations. This intermediary software layer bridges the gap between the simple RISC-V instruction set and complex AI inference requirements, making the system easier to manufacture and deploy by providing ready-to-use software support.

The patent applies preliminary action by pre-compiling and optimizing commonly used AI algorithms into RISC-V machine code, and by pre-defining instruction sequences for standard neural network operations such as convolution, activation functions, and pooling. This preliminary software preparation reduces the burden on end users and simplifies the manufacturing process by providing a ready-made software ecosystem.

The patent extracts the AI inference capability from the cloud environment and embeds it directly into terminal devices through a dedicated AI inference chip based on RISC-V architecture. This extraction eliminates the need for network communication during inference operations, thereby removing the network-induced delay while maintaining sufficient computing power for local AI processing.

The patent applies preliminary action by performing AI model training in the cloud beforehand, and then deploying the trained models to terminal devices. The RISC-V based AI inference chip at the terminal can then execute these pre-trained models locally without needing to communicate with the cloud during inference, thus achieving low-latency real-time processing while maintaining the benefits of cloud-based model development.