A Low-Latency Tightly Coupled Data Communication Method for Custom Functional Modules

Abstract

This application discloses a low-latency, tightly coupled data communication method for custom functional modules, relating to the field of processor technology. In the communication architecture employed by this method, a dedicated point-to-point low-latency communication interface is used for data communication between the memory access execution unit (LSU) and the custom functional module. This achieves tight coupling between the custom functional module and the processor module core. The processor module can communicate with the custom functional module via the LSU and the dedicated bus interface using memory access instructions. This method reduces the latency of the processor module accessing data from the custom functional module while preserving the independence of the custom functional module's functionality. It is flexible and simple to implement, requires minimal modification to the processor module but significantly improves performance, provides an optimized data path, ensures low-latency real-time requirements, and meets various scenarios with high performance and latency demands.

Description

Technical Field

[0001] This application relates to the field of processor technology, and in particular to a low-latency tightly coupled data communication method for a custom functional module. Background Technology

[0002] A CFPU (Custom Function Processing Unit) is a specially designed hardware processing unit designed to accelerate specific computational tasks or functions. Unlike general-purpose processors (such as CPUs or GPUs), CFPUs focus on performing specific, highly customized operations, typically offering significant advantages in performance, power consumption, or latency. CFPUs are commonly used in scenarios requiring efficient processing of specific tasks. High-speed, low-latency communication between the CFPU and CPU cores can lead to better performance in scenarios such as artificial intelligence (AI), machine learning (ML), signal processing, encryption / decryption, and image processing.

[0003] Currently, there are two main ways to implement CFPU:

[0004] (1) Embed it as an execution unit within the CPU microarchitecture pipeline. This implementation is primarily used for CFPUs with well-defined functions and high usage frequency, such as the ALU (Arithmetic and Logic Unit), MAC (Multiply-Accumulate Unit), Vector FU (Vector Functional Unit), and accelerators like the Gemmini DNN. Based on modern classic superscalar CPU microarchitectures, such as... Figure 1 The CFPU-1 in the example adopts this embedded structure. This embedded approach has a significant impact on the CPU, affecting everything from instruction decoding to execution and write-back. It is especially impactful on complex execution units with uncertain execution cycles, significantly affecting the CPU pipeline. Custom instructions, compilation tools, and software libraries are required to support its use. This results in a large workload and cost for this implementation method, poor architectural flexibility, complex implementation, and a long design and verification cycle. It also requires the support of custom instructions and compilation toolchains.

[0005] (2) Treat it as an independent functional module and use a system interconnect bus (such as the AXI bus) to communicate with the CPU core via read and write shared memory, thereby achieving loosely coupled interconnection. This is based on a modern classic superscalar CPU microarchitecture, such as... Figure 1The CFPU-2 in the example adopts this embedded structure, which is commonly used in current system-level IP units. Loose coupling is a common approach, but the addition of the system bus between the CFPU and the CPU, along with multiple master and slave ports, introduces significant data latency. The entire data transfer path is lengthy, requiring passage through the internal and external core buses and arbitration, resulting in considerable time and latency. For performance-critical applications, this implementation introduces substantial latency, and the inflexible data path becomes a performance bottleneck, difficult to optimize with limited room for improvement, thus restricting real-time performance and flexibility. Summary of the Invention

[0006] To address the aforementioned problems and technical requirements, this application proposes a low-latency, tightly coupled data communication method for custom functional modules. The technical solution of this application is as follows:

[0007] A low-latency, tightly coupled data communication method for a custom functional module, wherein the custom functional module is interconnected with the memory access execution unit (LSU) in the processor module using a point-to-point low-latency dedicated bus interface, the low-latency, tightly coupled data communication method comprising:

[0008] During the processor module's instruction pipeline execution, in the data acquisition phase of the Memory Access Execution Unit (LSU), based on the address space of the custom function module calculated in the address generation phase, the LSU uses the Load instruction to initiate a point-to-point non-cached read / write request to the custom function module via a point-to-point low-latency dedicated bus interface. After receiving the data returned by the custom function module, the LSU continues to execute subsequent pipeline steps, completing the data reading process for the custom function module.

[0009] Further technical solutions include: the low-latency tightly coupled data communication method also includes:

[0010] The processor module pre-configures the address space of the custom function module and sets the physical memory attribute (PMA) of the address space of the custom function module to Non-Cacheable. It then uses the Store instruction to configure the control status register (CSR) of the custom function module by writing to the memory register in the internal space of the custom function module.

[0011] Its further technical solution is that the low-latency tightly coupled data communication method is implemented based on a modern reduced instruction set architecture.

[0012] The further technical solution is that the data reading process uses the standard Load instruction of the modern reduced instruction set architecture to read data from the custom function module, and reads back one general-purpose register data bit width each time.

[0013] A further technical solution is that the data reading process uses a custom Load instruction of the modern reduced instruction set architecture to read data from the custom functional module, and performs multi-data prefetching by reading back multiple register data each time.

[0014] A further technical solution is that after receiving a point-to-point non-cached read / write request, the custom function module returns the data temporarily stored in the data storage RAM to the processor module. The data temporarily stored in the data storage RAM comes from the internal operation logic of the custom function module or is obtained from an external data source through the data interface.

[0015] A further technical solution is that the data returned by the custom function module is temporarily stored in the read buffer of the LSU and skips the cache to enter the subsequent pipeline steps. The data to be written to the custom function module by the processor module is temporarily stored in the write buffer of the LSU and skips the cache to be written to the custom function module.

[0016] A further technical solution involves interconnecting the memory access execution unit (LSU) with multiple custom functional modules using point-to-point low-latency dedicated bus interfaces.

[0017] The further technical solution is that the custom function module is a single circuit module or a system-level integrated module that integrates multiple circuit modules.

[0018] Its further technical solution is that the custom functional module supports the storage-computing integrated cache type.

[0019] A further technical solution is that the data reading process remains in a sleep state when it does not receive a data reading notification from the custom function module, and is awakened and begins execution after receiving a data reading notification from the custom function module.

[0020] The beneficial technical effects of this application are:

[0021] This application discloses a low-latency, tightly coupled data communication method for custom functional modules. This method proposes that custom functional modules can be moved inward to the processor module core and tightly coupled with the memory access execution unit (MLU) via a point-to-point low-latency dedicated bus interface. The processor module can communicate with the custom functional module using only memory access instructions through the LSU and the dedicated bus interface. This method reduces the latency of the processor module accessing data from the custom functional module while preserving the independence of the custom functional module's functionality. Compared to traditional embedded and loosely coupled methods, this method is flexible and simple to implement, requires minimal changes to the processor module, has low cost, does not affect the CPU pipeline, does not require instruction-level scheduling, does not maintain cache coherence, and is simple to implement. Implementation only requires modification of some interfaces in the hardware microarchitecture, resulting in minimal changes and a limited impact. Implementation is relatively easy and does not require modification of the software toolchain, yet it significantly improves performance and can fully utilize the CFPU, providing an optimized data path, ensuring low-latency real-time requirements, and meeting various scenarios with high performance and latency demands.

[0022] This method, implemented based on a modern reduced instruction set architecture (RISC), provides a third possibility for high-speed, low-latency communication between custom functional modules and processor modules. Benefiting from the characteristics of RISC, this method enables modular and customized instructions, allowing for multi-data prefetching by adding custom Load instructions. Only the corresponding custom Load instruction needs to be added to the ID decoding module for decoding; the backend execution only requires optimization at the data buffer at the interface, further improving data transfer efficiency and maximizing real-time performance.

[0023] This method can also add interconnect networks to expand the number of supported custom functional modules as needed and for performance requirements, achieving scalability and flexibility. In addition, this method is compatible with traditional architectures to achieve more flexible functional expansion. Attached Figure Description

[0024] Figure 1 This is a communication architecture diagram of the CFPU and CPU core using existing embedded and loosely coupled architectures.

[0025] Figure 2 This is a communication architecture diagram used by the custom function module and processor module in this application.

[0026] Figure 3 This is a flowchart illustrating a low-latency tightly coupled data communication method according to an embodiment of this application. Detailed Implementation

[0027] The specific embodiments of this application will be further described below with reference to the accompanying drawings.

[0028] This application discloses a low-latency, tightly coupled data communication method for a custom functional module. The data communication architecture adopted by this low-latency, tightly coupled data communication method is as follows: Figure 2 As shown. Figure 2 The following description of this embodiment will be used as an example. Unlike the traditional loosely coupled architecture, the architecture of this application moves the integration location of the custom function module from the traditional external system bus interface to the processor module. However, unlike the traditional embedded structure that embeds the custom function module as an execution unit into the pipeline, the custom function module moved into the processor module is interconnected with the Load & Store Unit (LSU) in the processor module using a point-to-point low-latency dedicated bus interface (represented by the IF interface in the figure), thereby achieving tight coupling between the interface of the custom function module and the LSU in the processor module.

[0029] based on Figure 2 The data communication architecture shown, and an embodiment of the low-latency tightly coupled data communication method, include the following steps, please refer to... Figure 3 The flowchart shown:

[0030] First, the processor module needs to be configured, which includes two parts: (1) The processor module pre-configures the address space of the custom function module and configures the physical memory attributes (PMA) of the address space of the custom function module as non-cached to avoid data consistency maintenance problems. (2) The Store instruction is used to configure the control status register (CSR) of the custom function module by writing to the memory register in the internal space of the custom function module.

[0031] After completing the processor module configuration above, custom function modules can be enabled. The processor module's data reading process can then use the Load instruction to read data from these custom function modules. Specifically:

[0032] During the execution of the processor module instruction pipeline, in the data acquisition phase (DC phase) of the Memory Access Execution Unit (LSU), based on the address space of the custom function module calculated in the address generation phase (AG phase), the LSU uses the Load instruction to initiate a point-to-point non-cached read / write request to the custom function module via the point-to-point low-latency dedicated bus interface. After receiving the data returned by the custom function module, the LSU continues to execute subsequent pipeline steps, thus completing the data reading process from the custom function module.

[0033] Upon receiving a point-to-point non-cached read / write request, the custom function module returns temporarily stored data in its data storage RAM to the processor module. This temporarily stored data originates from the custom function module's internal computational logic or is obtained from an external data source via a data interface. The custom function module's internal computational logic includes at least one of FPGA programmable logic, in-memory computing logic, or near-memory computing logic. External data sources for the custom function module include various sensors or external storage.

[0034] Figure 2 Taking a processor module using a typical CPU microarchitecture pipeline as an example, this method does not affect the instruction fetch (IF), decode (ID), and instruction issue (IS) stages when only standard instructions are used. Only the Load and Store instructions are used, and the impact only involves the internal microarchitecture implementation of the LSU (Least Memory Execution Unit). Figure 2 Taking a typical four-stage pipelined microarchitecture as an example, the LSU (Memory Access Execution Unit) includes an address generation stage (AG stage), a data acquisition stage (DC stage), a data alignment stage (DA stage), and a data write-back stage (WB stage). After receiving data from the custom function module, the LSU continues executing the data alignment stage (DA stage) and the data write-back stage (WB stage), updates the registers, and the program continues running. The data reading process then enters the next waiting state for data return from the custom function module.

[0035] Please refer to Figure 2The Memory Execution Unit (LSU) adds a point-to-point low-latency dedicated bus interface that interfaces with the read buffer (RdBuffer) and write buffer (WrBuffer). Data returned by the custom function module is temporarily stored in the LSU's read buffer and skips the cache to proceed to subsequent pipeline steps. Data from the processor module to be written to the custom function module is temporarily stored in the LSU's write buffer and skips the cache before being written to the custom function module. In other words, the LSU's read and write operations on custom function modules do not require caching. In modern CPU hardware architectures, to bridge the gap between high CPU processing speed and slow memory access speed, tiered caches are often added. However, in multi-CPU cores or multi-level caches, using a cache can lead to multiple data backups at the same memory address. Furthermore, due to the CPU's write-back strategy, inconsistencies may arise between these backups before data synchronization is complete. Therefore, this application does not use a cache, ensuring a unique data source and avoiding data inconsistencies caused by multiple backups in the cache.

[0036] To facilitate modularization and customization of instructions, this low-latency, tightly coupled data communication method is implemented based on a modern reduced instruction set architecture, such as RISC-V. When the data reading process uses the Load instruction to read data from a custom functional module, one approach is to use the standard Load instruction of a modern RISC architecture, reading back one general-purpose register data bit width at a time; for example, the RV64I can read back 64 bits of data at a time. To further improve performance, in another embodiment, the data reading process uses a custom Load instruction of a modern RISC architecture to read data from the custom functional module, performing multiple data prefetching by reading back multiple registers at a time, thereby improving bus utilization. For example, in one instance, a custom Load instruction is implemented using a RISC-V instruction set I-type custom instruction. In this case, the funct3 field of the custom Load instruction records the number of register data to be prefetched, and the opcode field is set to 0001011, indicating the use of the custom type.

[0037] In addition, to reduce power consumption, the data reading process remains in a sleep state when it does not receive a data read notification from the custom function module. Upon receiving the notification, it is awakened and begins executing the aforementioned method. The custom function module sends the data read notification to the processor module via interrupts or other means.

[0038] This method can expand the number of supported custom function modules. The memory execution unit (LSU) is interconnected with multiple custom function modules using point-to-point low-latency dedicated bus interfaces, thereby forming an interconnection network between the LSU and multiple custom function modules. It has good scalability and flexibility according to needs and performance requirements.

[0039] In one embodiment, the aforementioned custom function module is a single-circuit module, such as a commonly existing custom function processing unit (CFPU).

[0040] In another embodiment, the aforementioned custom functional module is a system-level integrated module that integrates multiple circuit modules. The number, type, and integration method of the circuit modules integrated within the custom functional module can be flexibly adjusted. The various forms of custom functional modules can be tightly coupled to the memory access execution unit (LSU) of the processor module using the method provided in this application, exhibiting strong flexibility and versatility. In one embodiment where the custom functional module adopts a system-level integrated module structure, the aforementioned custom functional module is a processor with an embedded CFPU. That is, in a multi-core scenario, the CFPU is embedded into the processor's pipeline using a traditional embedded structure and forms a custom functional module with the processor. In other words, the communication architecture provided in this application is compatible with traditional embedded architectures.

[0041] Because the method provided in this application achieves tight coupling between the custom function module and the memory access execution unit (LSU) of the processor module, the custom function module can provide data processing results for the processor module to perform scheduling and decision-making. In a typical application, regardless of whether a single-circuit module or a multi-circuit module integration is used, the custom function module supports in-memory compute cache type custom function module. The tight coupling and low-latency communication between this type of custom function module and the processor module shows high potential.

[0042] Furthermore, regardless of whether the custom function module adopts a single-circuit module or a system-level integrated module structure, after the custom function module implements a tight coupling interface with the memory access execution unit (LSU) of the processor module using the method provided in this application, the custom function module and the processor module can form a variety of communication architectures, including master-slave communication architecture and peer-to-peer communication architecture.

[0043] Compared to the embedded and loosely coupled approaches that differ in the interconnection and integration locations of processor modules and custom functional modules, this method proposes a third approach: the custom functional module and the processor module core are tightly coupled via an interface. Data communication between the custom functional module and the memory execution unit (LSU) is achieved through a dedicated point-to-point low-latency communication interface. This significantly reduces data access latency while maintaining the flexibility and independence of the custom functional module, without affecting the CPU pipeline, requiring no instruction-level scheduling, and not maintaining cache coherence, thus simplifying implementation. A simple comparison is provided: taking the custom functional module as a CFPU as an example, the traditional loosely coupled method requires the CFPU to write to DDR, notify the CPU to retrieve data, and the CPU to read the data from DDR. However, the method provided in this application eliminates the need to write the CFPU result back to DDR; the CPU can directly read the result from the CFPU's data storage RAM via a dedicated point-to-point low-latency bus interface. The flexibility of the CFPU implementation can be fully utilized to select the appropriate data path. Traditional embedded methods require instruction support, such as the Gemmini DNN accelerator, which requires several instructions with different functions to complete the scheduling. For memory-type instructions, cache consistency needs to be maintained on the Last Level Cache, which is relatively complex to implement. The method in this application only needs to modify some interfaces of the hardware microarchitecture. The changes are small, the impact is limited, and the implementation is relatively easy without modifying the software toolchain.

[0044] The above descriptions are merely preferred embodiments of this application, and this application is not limited to the above embodiments. It is understood that other improvements and variations that can be directly derived or conceived by those skilled in the art without departing from the spirit and concept of this application should be considered to be included within the protection scope of this application.

Claims

1. A low-latency, tightly coupled data communication method for a custom functional module, characterized in that, The custom function module and the memory access execution unit (LSU) in the processor module are interconnected using a point-to-point low-latency dedicated bus interface. The low-latency tightly coupled data communication method includes: During the execution of the processor module's instruction pipeline, in the data acquisition phase of the Memory Access Execution Unit (LSU), based on the address space of the custom function module calculated in the address generation phase, the LSU uses the Load instruction to initiate a point-to-point Non-Cacheable read request to the custom function module through the point-to-point low-latency dedicated bus interface. After receiving the data returned by the custom function module, the LSU continues to execute subsequent pipeline steps to complete the data reading process of the custom function module. The low-latency tightly coupled data communication method further includes: the processor module pre-configures the address space of the custom function module and configures the physical memory attribute (PMA) of the address space of the custom function module as a non-cached attribute, and uses the Store instruction to configure the control status register (CSR) of the custom function module by writing to the memory register in the internal space of the custom function module; the data returned by the custom function module is temporarily stored in the read buffer of the LSU and skips the cache to enter the subsequent pipeline steps, and the data to be written to the custom function module by the processor module is temporarily stored in the write buffer of the LSU and skips the cache to be written to the custom function module.

2. The low-latency tightly coupled data communication method according to claim 1, characterized in that, The low-latency tightly coupled data communication method is implemented based on a modern reduced instruction set architecture. The data reading process uses the standard Load instruction of the modern reduced instruction set architecture to read data from the custom functional module, and reads back one general-purpose register data bit width each time.

3. The low-latency tightly coupled data communication method according to claim 2, characterized in that, The data reading process uses custom Load instructions of the modern reduced instruction set architecture to read data from custom functional modules and performs multi-data prefetching by reading back data from multiple registers each time.

4. The low-latency tightly coupled data communication method according to claim 1, characterized in that, After receiving a point-to-point non-cached read request, the custom function module returns the data temporarily stored in the data storage RAM to the processor module. The data temporarily stored in the data storage RAM comes from the internal operation logic of the custom function module or is obtained from an external data source through the data interface.

5. The low-latency tightly coupled data communication method according to claim 1, characterized in that, The Memory Execution Unit (LSU) is interconnected with multiple custom function modules using point-to-point low-latency dedicated bus interfaces.

6. The low-latency tightly coupled data communication method according to claim 1, characterized in that, Custom function modules can be single-circuit modules or system-level integrated modules that integrate multiple circuit modules.

7. The low-latency tightly coupled data communication method according to claim 1, characterized in that, Custom functional modules support the storage-and-computation integrated cache type.

8. The low-latency tightly coupled data communication method according to claim 1, characterized in that, The data reading process remains in a sleep state when it does not receive a data reading notification from the custom function module, and is awakened and begins execution after receiving a data reading notification from the custom function module.

Images