Intelligent DSP processor architecture based on RISC-V extension

Abstract

The present invention relates to an intelligent DSP processor architecture based on RISC-V extensions. By dynamically switching the topology of hardware-reused parallel arrays, the same hardware unit can be switched to AI matrix multiplication or DSP complex operation mode in real time, realizing dynamic switching between AI matrix multiplication and DSP complex operation, avoiding idle resources of discrete architectures, achieving efficient collaboration between AI and DSP tasks, and improving hardware resource utilization. At the same time, single-cycle operation is achieved through the newly added dedicated instruction set of the RISC-V RVV extended instruction set, and a circular buffer manager is designed to eliminate software simulation overhead, effectively reducing the latency of AI / DSP mixed tasks and realizing high-performance AI / DSP mixed task collaborative processing.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent computing acceleration and relates to an intelligent DSP processor architecture based on RISC-V extension. Background Art

[0002] Edge computing devices must simultaneously handle AI inference (such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs)) and real-time signal processing (Fourier transforms (FFTs) and finite impulse response (FIRs). However, these traditional solutions face the following bottlenecks: For example, due to the limitations of discrete architectures, mainstream solutions utilize a standalone NPU + DSP multi-core architecture (such as the Arm Ethos NPU + Cortex-M DSP). AI inference and DSP tasks must exchange data through shared memory or buses, resulting in underutilized hardware resources. In tasks such as voice wake-up (DSP pre-processing + AI inference), data transfer consumes a high percentage of power and results in significant latency. Furthermore, the RISC-V RVV instruction set is inefficient. While native RVV instructions (such as vwadd and vfmacc) support vector operations, they lack optimization for hybrid AI / DSP tasks. Matrix multiplications must be broken down into multiple vector multiply-add loops, lacking efficient hardware and instruction support. Complex operations (such as FFT butterfly operations) require software-emulated real and imaginary component calculations, failing to meet high real-time requirements. Therefore, how to achieve high-performance AI / DSP hybrid task collaborative processing has become one of the current technical issues. Summary of the Invention

[0003] In response to the problems existing in the above-mentioned traditional technologies, the present invention proposes an intelligent DSP processor architecture based on RISC-V extension, which can achieve high-performance AI / DSP hybrid task collaborative processing.

[0004] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0005] Provides an intelligent DSP processor architecture based on RISC-V extensions, including vector launch queues, scoreboards, vector decode and launch logic, vector register file read and write control, execution units, vector register files, AI-DSP hybrid computing units, vector load-store data first-in-first-out queues, advanced consistency interfaces, and data caches;

[0006] The vector decoding and transmission logic are respectively connected to the vector transmission queue, vector register file read and write control and execution unit. The vector transmission queue is connected to the scoreboard. The vector register file is respectively connected to the vector register file read and write control, AI-DSP hybrid computing unit and vector load-store data first-in-first-out queue. The vector load-store data first-in-first-out queue is respectively connected to the advanced consistency interface and data cache. The execution unit is connected to the AI-DSP hybrid computing unit. The vector register file is a unified vector register file with a width of 256 bits, supports mixed data type storage, and has built-in data type marking and overflow protection logic.

[0007] The AI-DSP hybrid computing unit executes the VCFG instruction through the hardware mode register to dynamically switch the hardware interconnection topology of the 4×4 parallel array, and reuses the 4×4 parallel array to perform AI matrix multiplication and DSP complex multiplication and addition; the hardware interconnection topology includes full parallel interconnection and cross-interconnection topology. The AI-DSP hybrid computing unit triggers parallel array calculation through the VMMA instruction, and the AI-DSP hybrid computing unit triggers complex multiplication and addition calculation through the VCMAC instruction and automatically calculates the wrap address through the loop buffer manager. The VCFG instruction, VMMA instruction, VCRD instruction and VCMAC instruction are instructions of the RISC-V RVV extended instruction set.

[0008] One of the above technical solutions has the following advantages and beneficial effects:

[0009] The above-mentioned intelligent DSP processor architecture based on RISC-V extension dynamically switches the topology through hardware multiplexing parallel arrays, so that the same hardware unit can switch to AI matrix multiplication or DSP complex operation mode in real time, realizing dynamic switching of AI matrix multiplication and DSP complex operation, avoiding idle resources of discrete architectures, achieving efficient collaboration of AI and DSP tasks, and improving hardware resource utilization; at the same time, single-cycle operation is achieved through the newly added dedicated instruction set of RISC-V RVV extended instruction set, and a circular buffer manager is designed to eliminate software simulation overhead, effectively reducing the latency of AI / DSP mixed tasks, and realizing high-performance AI / DSP mixed task collaborative processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the conventional technology, the following briefly introduces the drawings required for use in the embodiments or the conventional technology descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0011] Figure 11 is a schematic diagram of the overall architecture of an intelligent DSP processor architecture based on RISC-V extension in one embodiment;

[0012] Figure 2 Schematic diagram of a U8 matrix multiplication compensation module in one embodiment;

[0013] Figure 3 FIG. 1 is a schematic diagram of a method for implementing quantization truncation in an embodiment. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the description of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0015] It should be noted that the reference to "embodiment" in this document means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The presentation of this phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It will be understood by those skilled in the art that the embodiments described herein may be combined with other embodiments. The term "and / or" used in the specification of the present invention and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0016] The following describes the implementation of the present invention in detail with reference to the accompanying drawings in the embodiments of the present invention.

[0017] Traditionally, the discrete CPU+AI+DSP architecture utilizes independent AI accelerators (systolic arrays) and DSP coprocessors, interconnected via a NoC bus. The AI ​​accelerator supports INT8 matrix multiplication (i.e., matrix multiplication of 8-bit signed integers (INT8)), while the DSP coprocessor provides FFT hardware acceleration. In this architecture, data between the AI ​​accelerator and DSP coprocessor must be transmitted via the NoC bus, resulting in high end-to-end latency and high NoC bus occupancy. High-IO (input and output) tasks and AI inference tasks require time-sharing multiplexing of the NoC bus. The AI ​​accelerator and DSP coprocessor cannot share computing resources (such as MAC units), resulting in high static power consumption and area overhead, leading to wasted hardware resources.

[0018] The RISC-V RVV extended AI acceleration solution improves AI inference efficiency by extending the RISC-V RVV instruction set, adding the matrix multiplication instruction VMMA, and integrating a 4×4 systolic array to support INT8 / FP16 precision. However, this traditional solution does not optimize complex operations and loop addressing, and DSP tasks still require software implementation, resulting in a lack of DSP support and high FFT latency. The 4×4 systolic array is a systolic array structure consisting of 16 processing elements (PEs) arranged in 4 rows and 4 columns. Each PE is connected and communicates data only with its neighbors. Data propagates through the array according to a certain pattern and rhythm, flowing from one PE to another like a pulse. The 4×4 systolic array offers excellent simplicity and regularity, with each PE being identical. This helps reduce design costs and improve scalability, and also facilitates the design and manufacture of very large-scale integrated circuits.

[0019] The operating principle of a 4×4 systolic array is as follows: Taking matrix multiplication as an example, suppose we want to calculate the product of two matrices, A and B. The elements of matrix A can be input sequentially by row into the first row processing unit of the systolic array, and the elements of matrix B can be input sequentially by column into the first column processing unit. Driven by the clock signal, the elements of matrix A flow to the right, and the elements of matrix B flow downward. Each processing unit multiplies the received elements of matrix A and matrix B and adds the result to the previous partial sum to obtain a new partial sum. After multiple clock cycles, the result of the matrix product is finally obtained at the processing unit in the lower right corner of the array. In this way, as data flows through the systolic array, all processing units process the data flowing through them simultaneously and in parallel, thereby achieving efficient matrix multiplication operations.

[0020] Another traditional technology is the dynamically reconfigurable DSP architecture, which uses a reconfigurable data path. The same hardware unit can be dynamically configured for AI matrix multiplication or DSP filtering calculation, improving utilization through time-division multiplexing. However, this architecture requires a customized instruction set and is incompatible with the RISC-V ecosystem. Hardware mode switching requires reconfiguration of data path interconnection, resulting in a high development threshold and high reconstruction delay.

[0021] In the design of embedded edge computing chips, this paper proposes a new RISC-V processor architecture that integrates AI and DSP acceleration for scenarios such as the intelligent Internet of Things and mobile terminals, and applies it to the design of intelligent DSP processors. SIMD stands for Single Instruction Multiple Data. RISC-V is an open standard instruction set architecture (ISA) that, with its modular design and collaborative community, has led a new trend in processor design. RISC-V RVV (RISC-V Vector) is a vector processing extension of the RISC-V instruction set architecture.

[0022] In one embodiment, Figure 1 As shown, an intelligent DSP processor architecture based on RISC-V extensions is provided, including a vector transmit queue, a scoreboard, vector decode and transmit logic, a vector register file read / write control, an execution unit, a vector register file, an AI-DSP hybrid computing unit, a vector load-store data first-in-first-out queue, an advanced consistency interface, and a data cache. The vector decode and transmit logic are respectively connected to the vector transmit queue, the vector register file read / write control, and the execution unit. The vector transmit queue is connected to the scoreboard. The vector register file is respectively connected to the vector register file read / write control, the AI-DSP hybrid computing unit, and the vector load-store data first-in-first-out queue. The vector load-store data first-in-first-out queue is respectively connected to the advanced consistency interface and the data cache. The execution unit is connected to the AI-DSP hybrid computing unit. The vector register file is a unified 256-bit vector register file that supports mixed data type storage and has built-in data type tagging and overflow protection logic. The AI-DSP hybrid computing unit executes the VCFG instruction through the hardware mode register to dynamically switch the hardware interconnection topology of the 4×4 parallel array, and reuses the 4×4 parallel array to perform AI matrix multiplication and DSP complex multiplication and addition; the hardware interconnection topology includes full parallel interconnection and cross-interconnection topology. The AI-DSP hybrid computing unit triggers parallel array calculation through the VMMA instruction, and the AI-DSP hybrid computing unit triggers complex multiplication and addition calculation through the VCMAC instruction and automatically calculates the wrap address through the loop buffer manager. The VCFG instruction, VMMA instruction, VCRD instruction and VCMAC instruction are instructions of the RISC-V RVV extended instruction set.

[0023] It can be understood that the processor architecture (hardware architecture) includes vector issue queue (VIQ), scoreboard (ScoreBoard), vector decode and issue logic (Vector Decode and Issue Logic), vector register file read / write control (VRF read / write control), execution unit (EX), vector register file (ie VRF, VectorRegisterFile, supports mixed precision data type storage), AI-DSP hybrid computing unit (AI&DSP EXT), vector load-store data first-in-first-out queue (Vector LS Data FIFO), advanced consistency interface (ACE) and data cache (Dcache).

[0024] The scoreboard is a hardware mechanism used to dynamically schedule instruction execution. It tracks instruction execution status and monitors operand availability to determine when instructions can be sent to the execution units, avoiding data and structural conflicts and increasing parallelism. The vector issue queue (VIQ) temporarily stores vector instructions waiting to be issued. It caches vector instructions retrieved from the instruction cache and, based on specific rules and conditions, sends them to the vector decode and issue logic (VDL) at the appropriate time. The VDL decodes and issues vector instructions, analyzes information such as the instruction operation type and operands, and determines when to issue the instruction to the appropriate execution unit based on status information from components such as the scoreboard. EX0 through EX3 represent the execution units, respectively. VRF0 through VRF3 represent the vector register files, respectively.

[0025] The AI-DSP hybrid computing unit reuses a 4×4 parallel array to support AI matrix multiplication (U8 / FP16) and DSP complex multiplication and accumulation (16-bit complex CMAC, a specific computing unit or operation in convolutional neural networks (CNNs) based on 16-bit complex operations. CMAC stands for Convolutional Multiply-Accumulate). Dynamically switching the hardware interconnection topology through hardware mode registers (functional modules implemented within the AI ​​accelerator and DSP coprocessor) allows the AI-DSP hybrid computing unit to be used to implement AI accelerators, primarily performing matrix multiplication and accumulation operations, as well as to implement DSP acceleration, primarily performing FFT operations. Both operations perform a large number of multiplications, thus reusing hardware resources (i.e., the 4×4 parallel array).

[0026] The Circular Buffer Manager (CBM), a module within the AI ​​accelerator and DSP coprocessor, maintains multiple circular buffers in hardware. The depth of the circular buffers is configurable (e.g., 64 to 1024 points) and supports zero-overhead circular addressing. The vector register file is 256 bits wide, supports mixed data types (e.g., AI: U8 / FP16; DSP: Q15 / Q31 complex numbers), and includes built-in data type tagging and overflow protection logic. Vector load-store data FIFO queues, advanced coherence interfaces, and data caches are traditional components of the RISC-V processor architecture. The hardware maintains the circular buffer manager's loop address pointers and automatically calculates wraparound addresses using the VCRD [buf_id], offset instruction, where offset is the address offset. For example, when sliding the FIR filter data window, the read address automatically jumps to the starting address after reaching the end of the buffer, eliminating the need for software modulo operations and supporting zero-overhead circular addressing.

[0027] The main instructions of the RISC-V RVV extended instruction set of the intelligent DSP processor architecture based on RISC-V extension can be shown in Table 1:

[0028] Table 1

[0029]

[0030] Among them, instructions can be edited by the user, decoded by the RISC-V decoder, and executed by components inside the AI ​​accelerator and DSP coprocessor. By adding VMMA instructions (the instructions will be decoded by the RISC-V vector decoder and executed by components inside the AI ​​accelerator and DSP coprocessor), parallel array calculations can be triggered in a single cycle, supporting hardware acceleration of the entire process of AI matrix multiplication. The addition of VCMAC instructions can support up to 16-point FFT butterfly operations at a time, and the hardware directly completes complex multiplication and addition. By adding VCRD instructions, the hardware automatically handles circular buffer address wraparound, eliminating software simulation operation overhead, and realizing automatic data type conversion and overflow protection mechanism of the vector register file (VRF).

[0031] The above-mentioned intelligent DSP processor architecture based on RISC-V extension dynamically switches the topology by hardware multiplexing parallel arrays, so that the same hardware unit can switch to AI matrix multiplication or DSP complex operation mode in real time, realizing dynamic switching of AI matrix multiplication and DSP complex operation, avoiding idle resources of discrete architecture, achieving efficient collaboration of AI and DSP tasks, and improving hardware resource utilization; at the same time, single-cycle operation is realized through the newly added dedicated instruction set of RISC-V RVV extended instruction set, and a circular buffer manager is designed to eliminate software simulation overhead, effectively reduce the latency of AI / DSP mixed tasks, and realize high-performance AI / DSP mixed task collaborative processing.

[0032] More specifically, when the AI-DSP hybrid computing unit is in AI mode (matrix multiplication), the 4×4 parallel array is interconnected in a fully parallel manner. First, the bias input matrix A (4×4) is multiplied by matrix B (4×4), and then the bias input matrix is ​​added (MAC, Multiply-Accumulate Unit). 64 parallel multiplications and additions are completed in a single cycle (i.e., U8×U8+I32→INT32), and the accumulated results are compensated by the existing U8 matrix multiplication compensation module (e.g., Figure 2 After that, the quantization truncation module (automatic truncation + ZeroPoint (zero point) compensation) outputs the U8 result (as shown in Figure 3 shown).

[0033] When the AI-DSP hybrid computing unit is in DSP mode (complex multiplication and addition), the 4×4 parallel array switches to a cross-connect topology through the hardware mode register, supporting the simultaneous calculation of 16 pairs of complex numbers (16-bit real part + 16-bit imaginary part); a 16-point FFT butterfly operation (i.e., radix-2 mode) is completed in a single cycle, and the cross-connect performs complex multiplication and addition (16-bit complex CMAC).

[0034] In one embodiment, the AI-DSP hybrid computing unit completes the bit reversal of FFT input / output in a single cycle through the bit reversal acceleration unit in the DSP complex multiplication and addition.

[0035] It can be understood that the bit reversal acceleration unit is used to complete the bit reversal rearrangement of FFT input / output in a single cycle. It is implemented using the underlying hardware inside the AI ​​accelerator and DSP coprocessor, and is used to rearrange the input data address according to the bit reversal rule (such as the index binary bit reversal of 8-point FFT). It is completed in a single cycle through a dedicated hardware network (the internal hardware of the AI ​​accelerator and DSP coprocessor); the instruction support for bit reversal acceleration is: calling the vector bit reversal instruction (that is, the VBITREV src, dst instruction, the destination operand dst is used to store the result of the instruction operation) to trigger the bit reversal operation, which takes 1 cycle (clock cycle), thereby further eliminating software simulation overhead.

[0036] In one embodiment, the above-mentioned intelligent DSP processor architecture based on RISC-V extension also includes a dynamic resource allocation controller for dividing the execution units and storage bandwidth according to task requirements when dynamically switching between AI matrix multiplication and DSP complex multiplication and addition.

[0037] Specifically, the dynamic resource allocation controller can switch the working mode through the VCFG instruction, and uniformly implement real-time adjustment of the working mode to support dynamic switching of AI / DSP tasks.

[0038] In one embodiment, the RISC-V RVV extended instruction set further includes a VFMT instruction for setting the vector register data type of the vector register file.

[0039] Specifically, the main instructions of the RISC-V RVV extended instruction set in this embodiment can be further shown in Table 2:

[0040] Table 2

[0041]

[0042] The VFMT instruction can automatically set the vector register data type of the vector register file, thereby further improving processing efficiency.

[0043] In one embodiment, the vector register file is provided with a VRF dedicated area for pre-storing AI quantization parameters and DSP filter coefficients; the AI ​​quantization parameters and DSP filter coefficients pre-stored in the VRF dedicated area are loaded into a 4×4 parallel array through a VLMMP instruction.

[0044] It is understandable that the intelligent DSP processor architecture based on RISC-V extension also adopts the following parameter preloading mechanism: AI quantization parameters (such as scale factor Scale / zero point ZeroPoint) and DSP filter coefficients can be pre-stored in the VRF dedicated area and loaded into the matrix multiplication hardware unit (such as 4×4 parallel array) through the VLMMP instruction.

[0045] By adopting a parameter preloading mechanism in conjunction with a vector register file (VRF), quantization parameters and filter coefficients are pre-stored in the hardware unit and can be directly called when switching modes, thus achieving μs-level mode switching and supporting real-time load response.

[0046] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0047] The above embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that a person skilled in the art would be able to make numerous modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of the present invention. Therefore, the scope of the present invention shall be determined by the appended claims.

Claims

1. An intelligent DSP processor architecture based on RISC-V extension, characterized in that: Includes vector launch queue, scoreboard, vector decode and launch logic, vector register file read and write control, execution unit, vector register file, AI-DSP hybrid computing unit, vector load-store data first-in-first-out queue, advanced consistency interface and data cache; The vector decoding and transmission logic are respectively connected to the vector transmission queue, vector register file read and write control and execution unit. The vector transmission queue is connected to the scoreboard. The vector register file is respectively connected to the vector register file read and write control, AI-DSP hybrid computing unit and vector load-store data first-in-first-out queue. The vector load-store data first-in-first-out queue is respectively connected to the advanced consistency interface and data cache. The execution unit is connected to the AI-DSP hybrid computing unit. The vector register file is a unified vector register file with a width of 256 bits, supports mixed data type storage, and has built-in data type marking and overflow protection logic. The AI-DSP hybrid computing unit executes the VCFG instruction through the hardware mode register to dynamically switch the hardware interconnection topology of the 4×4 parallel array, and reuses the 4×4 parallel array to perform AI matrix multiplication and DSP complex multiplication and addition; the hardware interconnection topology includes full parallel interconnection and cross-interconnection topology. The AI-DSP hybrid computing unit triggers parallel array calculation through the VMMA instruction, and the AI-DSP hybrid computing unit triggers complex multiplication and addition calculation through the VCMAC instruction and automatically calculates the wrap address through the loop buffer manager. The VCFG instruction, VMMA instruction, VCRD instruction and VCMAC instruction are instructions of the RISC-V RVV extended instruction set.

2. The RISC-V extended intelligent DSP processor architecture according to claim 1, characterized in that: The AI-DSP hybrid computing unit completes the bit reversal of FFT input / output in a single cycle through the bit reversal acceleration unit in the DSP complex multiplication and addition.

3. The intelligent DSP processor architecture based on RISC-V extension according to claim 1 or 2, characterized in that: It also includes a dynamic resource allocation controller, which is used to divide the execution units and storage bandwidth according to task requirements when dynamically switching between AI matrix multiplication and DSP complex multiplication and addition.

4. The intelligent DSP processor architecture based on RISC-V extension according to claim 3 is characterized in that: The RISC-VRVV extended instruction set also includes a VFMT instruction for setting the vector register data type of the vector register file.

5. The intelligent DSP processor architecture based on RISC-V extension according to claim 3 is characterized in that: The vector register file is provided with a VRF dedicated area for pre-storing AI quantization parameters and DSP filter coefficients; the AI ​​quantization parameters and DSP filter coefficients pre-stored in the VRF dedicated area are loaded into a 4×4 parallel array through a VLMMP instruction.

Images