On-chip intelligent memory controller and control method for heterogeneous computing system

Abstract

The invention provides an on-chip intelligent memory controller used in a heterogeneous computing system and a control method. The on-chip intelligent memory controller can receive instructions from the CPU and actively analyze and execute the instructions to carry out corresponding data processing and data transmission or configuration, so that instruction or data transmission or configuration operation originally needing to be executed between the CPU and other heterogeneous computing units is unloaded to the on-chip intelligent memory controller to be executed. And the workload of the CPU and the interaction frequency between the CPU and other heterogeneous computing units are reduced, so that the efficiency of cooperative work between the computing units in the heterogeneous computing system is improved.

Description

Technical Field

[0001] The present application generally relates to the field of heterogeneous computing, and more particularly to an on-chip intelligent memory controller and control method for use in a heterogeneous computing system. Background Art

[0002] In heterogeneous computing architectures, efficient data management and operations are required to ensure that multiple different types of heterogeneous computing units (e.g., central processing units (CPUs), graphics processing units (GPUs), neural network processing units (NPUs), or domain-specific accelerators (DSAs), etc.) work together efficiently. Existing heterogeneous computing systems usually only passively provide data access capabilities to on-chip storage hierarchies (e.g., L1 / L2 / L3 caches, synchronous random access memory (SRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, DDR for short), etc.), but these storage hierarchies cannot actively start and execute data processing operations based on instructions. In addition, existing cache consistency protocols are mainly designed for cache operations and are insufficient to support complex data operations in heterogeneous computing systems. Existing solutions all require modifying the functions of heterogeneous computing units in heterogeneous computing systems, which is very risky and costly for these computing units. Summary of the invention

[0003] In view of the above-mentioned problems, the present application provides an on-chip intelligent memory controller and a control method for a heterogeneous computing system.

[0004] According to one aspect of the present application, there is provided an on-chip intelligent memory controller for a heterogeneous computing system, the heterogeneous computing system comprising a central processing unit and one or more heterogeneous computing units, the on-chip intelligent memory controller comprising an instruction interface module, a cache consistency processing module, and a data processing module, wherein: the instruction interface module is used to: receive instructions from the central processing unit, parse the instructions and determine whether the instructions are associated with cacheable data, when it is determined that the instructions are associated with cacheable data, send the instructions to the cache consistency processing module, and when it is determined that the instructions are associated with non-cacheable data, send the instructions to the data processing module; the cache consistency processing module is used to: perform cache consistency processing on the instructions, and send the instructions to the data processing module after performing the cache consistency processing; the data processing module is used to: execute the instructions to perform corresponding data processing, and send the data processing results to the instruction interface module to transfer or configure the data associated with the instructions to one or more heterogeneous computing units.

[0005] According to another aspect of the present application, there is provided an intelligent memory control method for a heterogeneous computing system, the heterogeneous computing system comprising a central processing unit and one or more heterogeneous computing units, the intelligent memory control method comprising an on-chip intelligent memory controller arranged in the heterogeneous computing system performing the following operations: receiving an instruction from the central processing unit for parsing; determining whether the instruction is associated with cacheable data; when it is determined that the instruction is associated with cacheable data, performing cache consistency processing on the instruction, and after performing the cache consistency processing, executing the instruction to perform corresponding data processing; when it is determined that the instruction is associated with uncacheable data, executing the instruction to perform corresponding data processing; and based on the data processing result obtained by executing the instruction, delivering or configuring the data associated with the instruction to one or more heterogeneous computing units.

[0006] According to yet another aspect of the present application, a heterogeneous computing system is provided, comprising the on-chip intelligent memory controller as described above.

[0007] According to the on-chip intelligent memory controller and control method for heterogeneous computing systems of the present application, a specially configured on-chip intelligent memory controller can receive instructions from a CPU in the heterogeneous computing system, parse and execute the instructions to actively perform corresponding data processing and data transfer or configuration, thereby offloading instructions or data transfer or configuration operations that originally need to be executed between the CPU and other heterogeneous computing units (for example, GPU, NPU or DSA) to the on-chip intelligent memory controller for execution, thereby reducing the workload of the CPU and the number of interactions between the CPU and other heterogeneous computing units, and improving the efficiency of collaborative work between the computing units in the heterogeneous computing system. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The present application can be better understood from the following description of the specific embodiments of the present application in conjunction with the accompanying drawings, in which: Figure 1 A schematic logical block diagram of an example heterogeneous computing system including an on-chip smart memory controller according to some embodiments of the present application is shown; Figure 2 A schematic logic block diagram of an on-chip intelligent memory controller in a heterogeneous computing system according to some embodiments of the present application is shown; Figure 3 A schematic operational flow chart of an on-chip smart memory controller for a heterogeneous computing system according to some embodiments of the present application is shown; Figure 4 An exemplary interaction process when executing instructions associated with cacheable data in a heterogeneous computing system according to some embodiments of the present application is shown; Figure 5An exemplary interaction process when executing instructions associated with non-cacheable data in a heterogeneous computing system according to some embodiments of the present application is shown; Figure 6 An example of a load-reserve (LR) instruction and a conditional store (SC) instruction for implementing an atomic operation in a fifth-generation reduced instruction set computer architecture (RISC-V) instruction set is shown; Figure 7 An example of performing atomic operations implemented by LR and SC instruction pairs and smart memory control instructions defined in the RISC-V instruction format in a heterogeneous computing system according to some embodiments of the present application is shown; Figure 8 An example of intelligent memory control instructions defined in the RISC-V instruction format according to some embodiments of the present application is shown; Fig. 9 An example of an atomic memory operation (AMO) instruction in the RISC-V instruction set for implementing atomic operations is shown; Fig.10 An example of performing atomic operations of AMO instructions in a heterogeneous computing system according to some embodiments of the present application is shown; Fig.11 An example of an interactive process for implementing a CPU configuration operation on a GPU register in a traditional heterogeneous computing system is shown; Fig.12 An example of an interactive process for implementing a configuration operation of a CPU on a register of a GPU in a heterogeneous computing system according to some embodiments of the present application is shown. DETAILED DESCRIPTION

[0009] The features and exemplary embodiments of various aspects of the application will be described in detail below. In the detailed description below, many specific details are proposed to provide a comprehensive understanding of the application. However, it is obvious to those skilled in the art that the application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the application by illustrating the example of the application. The application is by no means limited to any specific configuration proposed below, but covers any modification, replacement and improvement of elements, parts and algorithms without departing from the spirit of the application. In the accompanying drawings and the following description, known structures and technologies are not shown to avoid causing unnecessary ambiguity to the application.

[0010] Furthermore, various operations will be described as multiple discrete operations in a manner that is most helpful for understanding the illustrative embodiments; however, the order of description should not be construed as implying that these operations are necessarily order dependent. In particular, these operations do not need to be performed in the order presented.

[0011] "Embodiments" and "some embodiments" are mentioned herein, but it should be understood that the features mentioned in each embodiment are not necessarily applicable only to that embodiment, but may be used in other embodiments. Features in one embodiment may be applied to another embodiment, or may be included in another embodiment. In addition, unless the context dictates otherwise, the terms "comprising", "having" and "including" are synonymous. The phrases "A or B" and "A / B" mean "(A), (B) or (A and B)".

[0012] The on-chip storage hierarchy in traditional heterogeneous computing systems (e.g., L1 / L2 / L3 cache, SRAM, DDR, etc.) usually only provides data access capabilities passively and cannot initiate and execute data operations based on instructions. In addition, the existing cache consistency protocols are mainly designed for cache operations and are not sufficient to support complex data operations in heterogeneous computing systems. Existing solutions all require modifying the functions of computing units in heterogeneous computing systems, which is very risky and costly for these computing units.

[0013] According to an embodiment of the present application, it is proposed to specially set up an on-chip smart memory controller (SMC) in a heterogeneous computing system. The SMC is coupled to each heterogeneous computing unit (such as CPU, GPU, NPU, DSA, etc.) in the heterogeneous computing system, for example, through a cache consistency structure bus. The SMC can receive instructions from the CPU, parse and execute instructions to actively perform corresponding data processing and data transfer or configuration, thereby unloading instructions or data transfer or configuration operations that originally need to be executed between the CPU and other heterogeneous computing units to the on-chip smart memory controller for execution, reducing the workload of the CPU and the number of interactions between the CPU and other heterogeneous computing units, thereby improving the efficiency of collaborative work between the computing units in the heterogeneous computing system.

[0014] On the other hand, the on-chip SMC according to the embodiment of the present application can perform cache consistency processing on instructions associated with cacheable data based on the existing cache consistency protocol, and then execute the instructions to perform corresponding data processing and subsequent data transfer or configuration operations, or can directly execute instructions associated with non-cacheable data to perform corresponding data processing and subsequent data transfer or configuration operations. The on-chip SMC can support the transfer or configuration of cacheable data and non-cacheable data at the same time, so that there is no need to modify the functions of the computing units in the heterogeneous computing system to support complex data operations in the heterogeneous computing system.

[0015] Figure 1 1 shows a schematic logic block diagram of an example heterogeneous computing system including an on-chip smart memory controller (SMC) according to some embodiments of the present application. Figure 1 As shown, the heterogeneous computing system includes various heterogeneous computing units (such as CPU subsystem, GPU subsystem, NPU subsystem), storage hierarchy (such as L2, L3 cache, DDR, system-level buffer (SLB)), cache consistency structure bus, and PCIe interface module for communicating with peripheral devices in traditional heterogeneous computing systems. In addition to these traditional components in traditional heterogeneous computing systems, the heterogeneous computing system according to an embodiment of the present application also includes a specially set on-chip SMC. Figure 1 As shown, the on-chip SMC and SLB together can be referred to as an on-chip memory controller subsystem. The on-chip SMC can be coupled to various heterogeneous computing units and various storage hierarchical structures in a heterogeneous computing system through a cache consistency structure bus. Figures 2 to 5 The structure and working process of the on-chip SMC are described in detail.

[0016] Figure 2 FIG. 1 shows a schematic logic block diagram of an on-chip SMC in a heterogeneous computing system according to some embodiments of the present application. Figure 2 As shown, the on-chip SMC can be coupled with a system-level buffer (SLB, usually SRAM) to form a Figure 1 The on-chip memory controller subsystem shown in . The on-chip SMC includes three modules: an instruction interface module, a cache consistency module, and a data processing module. The instruction interface module is used to receive instructions from the CPU, parse the instructions and determine whether the instructions are associated with cacheable data, and when it is determined that the instruction is associated with cacheable data, send the instruction to the cache consistency processing module, and when it is determined that the instruction is associated with non-cacheable data, send the instruction to the data processing module. The cache consistency processing module is used to perform cache consistency processing on the instruction, and send the instruction to the data processing module after performing the cache consistency processing. The data processing module is used to execute instructions to perform corresponding data processing, and send the data processing results to the instruction interface module to pass or configure the data associated with the instruction to one or more heterogeneous computing units. The following will refer to Figure 3 The working process of the on-chip SMC is described in detail and will be referred to Figure 4 and Figure 5 An exemplary interaction process between an on-chip SMC and other components in a heterogeneous computing system is described in detail.

[0017] Figure 3 FIG. 1 is a schematic operation flow chart of an on-chip SMC in a heterogeneous computing system according to some embodiments of the present application. Figure 3As shown, the instruction interface module can receive instructions from the central processing unit and parse the instructions, and then determine whether the instruction is associated with cacheable data. When it is determined that the instruction is associated with cacheable data, the instruction interface module can send the instruction to the cache consistency processing module for cache consistency processing, and the cache consistency processing module sends the instruction to the data processing module after performing cache consistency processing; when it is determined that the instruction is associated with non-cacheable data, the instruction interface module can directly send the instruction to the data processing module. The data processing module can execute the instruction to perform corresponding data processing, and send the data processing result to the instruction interface module, and pass or configure the data associated with the instruction to one or more heterogeneous computing units through the instruction interface module. When the SMC finishes executing the instruction, it can send a confirmation of the completion of the instruction execution to the central processing unit. In addition, before executing the instruction, the data processing module can determine whether the data associated with the instruction needs to be processed in the SLB, and when it is determined to be necessary, instruct the SLB to perform the corresponding data processing first. In addition, according to some embodiments of the present application, when performing cache consistency processing, the cache consistency processing module may also determine whether the data associated with the instruction needs to be processed in the SLB, and when it is determined to be necessary, instruct the SLB to perform the corresponding data processing.

[0018] Figure 4 An exemplary interaction process when executing instructions associated with cacheable data in a heterogeneous computing system according to some embodiments of the present application is shown. Figure 4 As shown, RN-F0, RN-F1, and RN-F2 represent three fully consistent request nodes. For example, RN-F0 may represent a CPU core in a CPU subsystem in a heterogeneous computing system, which sends instructions (e.g., RISC-V instructions) associated with cacheable data to the SMC to transfer or configure data to the GPU or NPU. When the SMC executes instructions associated with cacheable data, it is necessary to first perform cache consistency processing. For example, the SMC may need to interact with two related CPU cores (e.g., represented by RN-F1 and RN-F2) to perform cache consistency processing, such as processing based on a snoop consistency protocol. Then, the SMC can parse and execute the RISC-V instructions. When the SMC executes the RISC-V instructions, it may also need to interact with the SLB or DDR to perform memory access operations. After the SMC executes the RISC-V instructions, it may send an instruction execution completion confirmation to RN-F0, and transfer or configure data associated with the RISC-V instructions to the heterogeneous computing unit (GPU or NPU).

[0019] Figure 5An exemplary interaction process when executing instructions associated with non-cacheable data in a heterogeneous computing system according to some embodiments of the present application is shown. Figure 4 Similarly, SMC can parse and execute RISC-V instructions from RN-F0, and may need to interact with SLB or DDR to perform memory access operations when executing RISC-V instructions. After SMC executes the RISC-V instruction, it can send an instruction execution completion confirmation to RN-F0, and transfer or configure data associated with the RISC-V instruction to heterogeneous computing units such as GPU or NPU. Figure 5 and Figure 4 The only difference is that the RSIC-V instruction sent by RN-F0 to SMC is associated with uncacheable data, so SMC does not need to perform cache consistency processing and can directly execute the RSIC-V instruction.

[0020] In general, according to an embodiment of the present application, an on-chip SMC and a corresponding intelligent memory control method for a heterogeneous computing system are provided. The intelligent memory control method includes the following operations performed by the on-chip SMC: receiving instructions from the CPU for parsing; determining whether the instructions are associated with cacheable data; when it is determined that the instructions are associated with cacheable data, performing cache consistency processing on the instructions, and after performing cache consistency processing, executing the instructions to perform corresponding data processing; when it is determined that the instructions are associated with non-cacheable data, executing the instructions to perform corresponding data processing; and based on the data processing results obtained by executing the instructions, transferring or configuring the data associated with the instructions to one or more heterogeneous computing units. By using the on-chip SMC and the corresponding intelligent memory control method according to the embodiments of the present application, the instruction or data transfer or configuration operations that originally need to be executed between the CPU and other heterogeneous computing units can be offloaded to the SMC for execution, reducing the workload of the CPU and the number of interactions between the CPU and other heterogeneous computing units, thereby improving the efficiency of collaborative work between the computing units in the heterogeneous computing system.

[0021] According to an embodiment of the present application, the CPU can operate based on various instruction sets, and in particular, can operate based on the RSIC-V instruction set (including its extended instruction set) that is currently commonly used. Accordingly, according to an embodiment of the present application, the SMC can support the execution of various RSIC-V instructions in the RSIC-V instruction set (including its extended instruction set). In some embodiments of the present application, the SMC can support atomic operations implemented using load-reserve (LR) instructions and conditional store (SC) instructions.

[0022] Figure 6 Examples of LR instructions and SC instructions in the RISC-V instruction set for implementing atomic operations are shown. Figure 7An example of executing atomic operations implemented by LR and SC instruction pairs and smart memory control instructions defined in the RISC-V instruction format in a heterogeneous computing system according to some embodiments of the present application is shown.

[0023] The LR instruction is used to load the value of a memory address and "reserve" this address, that is, for a period of time thereafter, other processors' access to this address will be blocked until the paired SC instruction is executed. For example, assuming that in a traditional heterogeneous computing system, the CPU needs to perform N interactions with the GPU between the LR instruction and the SC instruction pair to complete a certain atomic operation, then the CPU generally needs to perform 2+N interactions with the GPU to complete the atomic operation. In contrast, in a heterogeneous computing system according to an embodiment of the present application, the CPU will only need to perform 3 interactions with the SMC (including LR, SC, and SMC_OP) to complete the atomic operation, and subsequent operations will be completed by the SMC interacting with the GPU. In this way, the ratio of the number of interactions required by the CPU when using SMC to the number of interactions required by the CPU when not using SMC is 3 / (2+N). Generally, the more data that needs to be transferred or configured, the more bandwidth efficiency is improved. Taking a GPU with 8 GPU cores or 128 vector engines as an example, the ratio is 30%, which means that the number of CPU interactions and related power consumption can be reduced by 70% when using SMC. When the GPU has more cores, the number of CPU interactions and related power consumption will be reduced more. For example, when the GPU has 16 GPU cores, the number of CPU interactions and related power consumption can be reduced by 83.3%.

[0024] Figure 8 An example of intelligent memory control instructions defined in the RISC-V instruction format according to some embodiments of the present application is shown. Figure 8 As shown, multiple SMC operations can be defined according to the RISC-V instruction format, and their operation codes can be represented by SMO.

[0025] In addition to the LR / SC instruction pair, the RISC-V instruction set also provides atomic memory operation (AMO) instructions to implement more direct and efficient atomic operations. Fig. 9 An example of an atomic memory operation (AMO) instruction in the RISC-V instruction set for implementing atomic operations is shown. Fig.10 An example of performing atomic operations of AMO instructions in a heterogeneous computing system according to some embodiments of the present application is shown. Fig.10The AMO instructions shown can be executed in the SMC, that is, the data processing, data transfer or configuration related to the AMO instruction can be completed through the interaction between the SMC and heterogeneous computing units such as GPU or NPU, which can greatly reduce the number of interactions between the CPU and these heterogeneous computing units. In addition, when these AMO instructions are executed by the SMC, they are not limited by the CPU cache line, and the granularity of data operations can be extended from cache lines to pages or even data blocks of any size. Moreover, since the SMC is closer to the data storage location (such as SLB) relative to the CPU core, executing these AMO instructions in the SMC may also reduce data operation time and latency.

[0026] In addition to the atomic operation instructions in the RISC-V instruction set described above, the heterogeneous computing system including the intelligent memory controller according to some embodiments of the present application may also support other new extended intelligent memory control instructions for memory operations defined in the RISC-V instruction format. For example, the configuration operation of the CPU on the registers in the GPU or the weight copy operation of the CPU on the NPU may be supported. The following takes the configuration operation of the CPU on the registers in the GPU as an example to further illustrate the improvement of the heterogeneous computing system according to the embodiment of the present application relative to the traditional heterogeneous computing system.

[0027] Fig.11 An example of an interactive process for implementing a CPU's configuration operation on a GPU's registers in a traditional heterogeneous computing system is shown. Fig.12 An example of an interactive process for implementing a configuration operation of a CPU on a register of a GPU in a heterogeneous computing system according to some embodiments of the present application is shown.

[0028] like Fig.11As shown, the CPU needs N interactions with the GPU to complete the configuration of the N registers in the GPU. In the heterogeneous computing system according to the embodiment of the present application, the CPU only needs to send an SMC instruction indicating the register configuration operation to the SMC, and the SMC will parse the instruction and perform subsequent related data processing and register configuration operations. In other words, after receiving the SMC instruction from the CPU, the SMC will take over the data processing and register configuration operations associated with the SMC instruction as the host. Therefore, the number of interactions required by the CPU will be reduced from N to 2, and the ratio of the number of interactions required by the CPU in the case of using SMC to the number of interactions required by the CPU in the case of not using SMC is 2 / N. The more registers the CPU needs to configure, the more bandwidth efficiency is improved. Similarly, taking a GPU with 8 GPU cores or 128 vector engines as an example, the ratio is 25%, which means that the number of CPU interactions and related power consumption can be reduced by 75% when SMC is used. When the GPU has more cores, the number of CPU interactions and related power consumption will be reduced more. For example, when the GPU has 16 GPU cores, the number of CPU interactions and related power consumption can be reduced by 87.5%.

[0029] In addition, a heterogeneous computing system including an intelligent memory controller according to an embodiment of the present application may also support various other data operation instructions as needed, such as a broadcast data (BDC) instruction for sending the same data from the SRAM to multiple destinations simultaneously; a multicast data (MCD) instruction, which is similar to the BDC instruction but allows data to be sent to a specified group of computing units; a data query (DQY) instruction, which allows querying the properties of data stored in the SRAM, such as checking the data type, size, or content-based query; a checksum and hash (CSH) instruction, which is used to generate checksums and hash values ​​directly from data stored in the SRAM; and the like.

[0030] In general, in the heterogeneous computing system according to the embodiment of the present application, by setting up an on-chip SMC to offload the instruction or data transfer or configuration operation that originally needs to be executed between the CPU and other heterogeneous computing units to the on-chip SMC for execution, the CPU workload and the number of interactions between the CPU and other heterogeneous computing units are reduced, and the efficiency of the collaborative work between the computing units in the heterogeneous computing system is improved. In addition, the on-chip SMC can simultaneously support the transfer or configuration of cacheable data and non-cacheable data, so that there is no need to modify the functions of the computing units in the heterogeneous computing system to support complex data operations in the heterogeneous computing system.

[0031] The following paragraphs describe examples of various embodiments of the present application.

[0032] Example 1 includes an on-chip intelligent memory controller for use in a heterogeneous computing system, the heterogeneous computing system includes a central processing unit and one or more heterogeneous computing units, the on-chip intelligent memory controller includes an instruction interface module, a cache consistency processing module, and a data processing module, wherein: the instruction interface module is used to: receive instructions from the central processing unit, parse the instructions and determine whether the instructions are associated with cacheable data, when it is determined that the instructions are associated with cacheable data, send the instructions to the cache consistency processing module, and when it is determined that the instructions are associated with non-cacheable data, send the instructions to the data processing module; the cache consistency processing module is used to: perform cache consistency processing on the instructions, and send the instructions to the data processing module after performing the cache consistency processing; the data processing module is used to: execute the instructions to perform corresponding data processing, and send the data processing results to the instruction interface module to transfer or configure the data associated with the instructions to one or more heterogeneous computing units.

[0033] Example 2 includes an on-chip intelligent memory controller according to Example 1, wherein the on-chip intelligent memory controller is coupled to a system-level buffer in a heterogeneous computing system, and, before executing an instruction, the data processing module is also used to: determine whether data associated with the instruction needs to be processed in the system-level buffer, and when it is determined to be necessary, instruct the system-level buffer to perform corresponding data processing.

[0034] Example 3 includes an on-chip intelligent memory controller according to Example 1 or 2, wherein, when performing cache consistency processing, the cache consistency processing module is also used to: determine whether data associated with the instruction needs to be processed in a system-level buffer, and when it is determined to be necessary, instruct the system-level buffer to perform corresponding data processing.

[0035] Example 4 includes an on-chip smart memory controller according to any of Examples 1 to 3, wherein the central processing unit includes one or more central processing unit cores based on the RISC-V instruction set architecture.

[0036] Example 5 includes an on-chip smart memory controller according to any one of Examples 1 to 4, wherein the instructions include at least one of the following items: a smart memory control instruction executed between a load-reserve LR and a conditional store SC instruction pair defined in the RISC-V instruction format; an atomic memory operation AMO instruction in the RISC-V instruction set; or a new extended smart memory control instruction defined in the RISC-V instruction format.

[0037] Example 6 includes an on-chip smart memory controller according to any of Examples 1 to 5, wherein the one or more heterogeneous computing units include one or more of the following: a graphics processing unit, a neural network processing unit, or a domain-specific accelerator.

[0038] Example 7 includes an on-chip smart memory controller according to any of Examples 1 to 6, wherein the instruction interface module is coupled to the central processing unit and the one or more heterogeneous computing units via a cache coherent fabric bus.

[0039] Example 8 includes an intelligent memory control method for a heterogeneous computing system, the heterogeneous computing system includes a central processing unit and one or more heterogeneous computing units, the intelligent memory control method includes an on-chip intelligent memory controller arranged in the heterogeneous computing system performing the following operations: receiving an instruction from the central processing unit for parsing; determining whether the instruction is associated with cacheable data; when it is determined that the instruction is associated with cacheable data, performing cache consistency processing on the instruction, and after performing the cache consistency processing, executing the instruction to perform corresponding data processing; when it is determined that the instruction is associated with uncacheable data, executing the instruction to perform corresponding data processing; and based on the data processing result obtained by executing the instruction, passing or configuring the data associated with the instruction to one or more heterogeneous computing units.

[0040] Example 9 includes an intelligent memory control method according to Example 8, wherein an on-chip intelligent memory controller is coupled to a system-level buffer in a heterogeneous computing system, and before executing an instruction, the intelligent memory control method further includes: determining whether data associated with the instruction needs to be processed in the system-level buffer, and when it is determined to be necessary, instructing the system-level buffer to perform corresponding data processing.

[0041] Example 10 includes an intelligent memory control method according to Example 8 or 9, wherein the intelligent memory control method further includes: when performing cache consistency processing, determining whether data associated with the instruction needs to be processed in a system-level buffer, and when it is determined that it is necessary, instructing the system-level buffer to perform corresponding data processing.

[0042] Example 11 includes an intelligent memory control method according to any one of Examples 8 to 10, wherein the central processing unit includes one or more central processing unit cores based on the RISC-V instruction set architecture.

[0043] Example 12 includes an intelligent memory control method according to any one of Examples 8 to 11, wherein the instruction includes at least one of the following items: an intelligent memory control instruction executed between a load-reserve LR and a conditional store SC instruction pair defined in the RISC-V instruction format; an atomic memory operation AMO instruction in the RISC-V instruction set; or a new extended intelligent memory control instruction defined in the RISC-V instruction format.

[0044] Example 13 includes an intelligent memory control method according to any of Examples 8 to 12, wherein the one or more heterogeneous computing units include one or more of the following: a graphics processing unit, a neural network processing unit, or a domain-specific accelerator.

[0045] Example 14 includes a heterogeneous computing system including an on-chip smart memory controller according to any of Examples 1-7.

[0046] Although certain embodiments have been illustrated and described herein for purposes of description, the embodiments illustrated and described may be replaced with a variety of alternative and / or equivalent embodiments or implementations that achieve the same purpose without departing from the scope of the present disclosure. This application is intended to cover any adaptation or variation of the embodiments discussed herein. Therefore, the embodiments described herein are expressly limited only by the appended claims and their equivalents.

Claims

1. An on-chip intelligent memory controller for a heterogeneous computing system, the heterogeneous computing system comprising a central processing unit and one or more heterogeneous computing units, the on-chip intelligent memory controller comprising an instruction interface module, a cache consistency processing module, and a data processing module, wherein: The instruction interface module is used to: receive an instruction from the central processing unit, parse the instruction and determine whether the instruction is associated with cacheable data, send the instruction to the cache consistency processing module when it is determined that the instruction is associated with cacheable data, and send the instruction to the data processing module when it is determined that the instruction is associated with non-cacheable data; The cache consistency processing module is used to: perform cache consistency processing on the instruction, and send the instruction to the data processing module after performing the cache consistency processing; The data processing module is used to: execute the instruction to perform corresponding data processing, and send the data processing result to the instruction interface module to transfer or configure the data associated with the instruction to the one or more heterogeneous computing units.

2. The on-chip smart memory controller according to claim 1, wherein: The on-chip intelligent memory controller is coupled to a system-level buffer in the heterogeneous computing system, and, before executing the instruction, the data processing module is further configured to: determining whether data associated with the instruction needs to be processed in the system level buffer, and When it is determined to be necessary, the system level buffer is instructed to perform corresponding data processing.

3. The on-chip smart memory controller according to claim 2, wherein: When executing the cache consistency processing, the cache consistency processing module is further used to: determining whether data associated with the instruction needs to be processed in the system level buffer, and When it is determined to be necessary, the system level buffer is instructed to perform corresponding data processing.

4. The on-chip smart memory controller according to claim 1, wherein: The central processing unit includes one or more central processing unit cores based on the RISC-V instruction set architecture.

5. The on-chip smart memory controller according to claim 1, wherein: The instructions include at least one of the following: Smart memory control instructions executed between load-reserve LR and conditional store SC instruction pairs as defined in the RISC-V instruction format; The Atomic Memory Operation (AMO) instruction in the RISC-V instruction set; or New extended intelligent memory control instructions defined in the RISC-V instruction format.

6. The on-chip smart memory controller according to claim 1, wherein: The one or more heterogeneous computing units include one or more of the following: a graphics processing unit, a neural network processing unit, or a domain-specific accelerator.

7. The on-chip smart memory controller according to any one of claims 1 to 6, wherein: The instruction interface module is coupled to the central processing unit and the one or more heterogeneous computing units through a cache coherence structure bus.

8. A smart memory control method for a heterogeneous computing system, the heterogeneous computing system comprising a central processing unit and one or more heterogeneous computing units, the smart memory control method comprising performing the following operations by an on-chip smart memory controller provided in the heterogeneous computing system: receiving instructions from the central processing unit for parsing; determining whether the instruction is associated with cacheable data; When it is determined that the instruction is associated with cacheable data, performing cache coherence processing on the instruction, and after performing the cache coherence processing, executing the instruction to perform corresponding data processing; When it is determined that the instruction is associated with non-cacheable data, executing the instruction to perform corresponding data processing; and Based on the data processing result obtained by executing the instruction, the data associated with the instruction is delivered or configured to the one or more heterogeneous computing units.

9. The intelligent storage control method according to claim 8, wherein: The on-chip intelligent memory controller is coupled to a system-level buffer in the heterogeneous computing system. Before executing the instruction, the intelligent memory control method further includes: determining whether data associated with the instruction needs to be processed in the system level buffer, and When it is determined to be necessary, the system level buffer is instructed to perform corresponding data processing.

10. The intelligent storage control method according to claim 9, wherein: The intelligent memory control method further includes: when executing the cache coherence processing, determining whether the data associated with the instruction needs to be processed in the system-level buffer, and when it is determined that it is necessary, instructing the system-level buffer to perform corresponding data processing.

11. The intelligent storage control method according to claim 8, wherein: The central processing unit includes one or more central processing unit cores based on the RISC-V instruction set architecture.

12. The intelligent storage control method according to claim 8, wherein: The instructions include at least one of the following: Smart memory control instructions executed between load-reserve LR and conditional store SC instruction pairs as defined in the RISC-V instruction format; The Atomic Memory Operation (AMO) instruction in the RISC-V instruction set; or New extended intelligent memory control instructions defined in the RISC-V instruction format.

13. The intelligent storage control method according to any one of claims 8 to 12, wherein: The one or more heterogeneous computing units include one or more of the following: a graphics processing unit, a neural network processing unit, or a domain-specific accelerator.

14. A heterogeneous computing system comprising an on-chip intelligent memory controller according to any one of claims 1 to 7.

Images