Instruction processing unit, instruction execution method, system-on-chip and board
By using the SaaP architecture and MS instruction set, the problems of low programming productivity and low hardware utilization caused by hardware heterogeneity are solved, enabling efficient operation on SoC and simplifying the programming framework, thereby improving hardware management efficiency.
Patent Information
- Application Number
- CN202210764246.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-29
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2042-06-29
AI Technical Summary
The low programming productivity and low hardware utilization caused by hardware heterogeneity in current computing systems make it difficult to run efficiently on different SoCs. Programming frameworks require a lot of development work and are difficult to support the heterogeneity of different IPs, resulting in high control and data exchange overhead.
It provides a unified system-on-chip architecture framework (SaaP) that adopts a hybrid-scale instruction set and on-chip interconnect, manages heterogeneous IP cores through a system controller, hides hardware heterogeneity, and uses MS instructions to implement a unified instruction format at the hardware level, supporting high productivity and hardware utilization.
It improves programming productivity and hardware utilization, reduces control and data exchange overhead, and enables efficient operation on different SoCs and simplifies the programming framework.
Smart Images

Figure CN117348930B_ABST
Abstract
Description
Technical Field
[0001] This disclosure generally relates to the field of instruction sets. More specifically, this disclosure relates to an instruction processing apparatus, an instruction execution method, a system-on-a-chip, and a circuit board. Background Technology
[0002] With the end of Moore's Law and Dennard Scaling, the performance gains of general-purpose CPUs have continued to decline. Domain-specific architectures (DSAs) have become the most promising and feasible way to continue improving the performance and efficiency of the entire computing system. DSAs have experienced a boom, and are considered to have ushered in a new golden age of computer architecture. A wide variety of DSAs have been proposed to accelerate specific applications, such as various xPUs, including DPUs for data stream processing, GPUs for image processing, NPUs for neural networks, TPUs for tensor processing, and so on. As more and more DSAs, especially DSAs for computing purposes (also known as IPs), are integrated into System-on-Chip (SoCs) to achieve high efficiency, the heterogeneity of hardware in current computing systems continues to grow, shifting from standardization to customization.
[0003] Currently, IP typically only exposes its associated hardware interfaces, forcing SoCs to manage the IP as a standalone device using code running on the host CPU. Because managing hardware heterogeneity directly for application developers is extremely difficult, significant effort is usually invested in building programming frameworks to help them manage this hardware heterogeneity. For example, popular programming frameworks for deep learning include PyTorch, TensorFlow, and MXNet, all of which provide application developers with high-level, easy-to-use Python interfaces.
[0004] Unfortunately, this heterogeneity in software management within CPU-centric SoCs hinders the efficient operation of user applications across different SoCs due to low productivity and hardware utilization. Low productivity stems from both programming frameworks and applications. For programming framework developers, supporting different SoCs requires implementing their respective high-level abstractions using different IPs, necessitating significant development work. For application developers, the differences between IPs within the SoC require different implementations for the same application, resulting in a heavy programming burden. Furthermore, the situation can be even worse for IPs not supported by the programming framework, as hardware heterogeneity needs to be manually managed. Low hardware utilization is related to CPU-centric SoCs and somewhat general-purpose IPs. In current SoCs, the host CPU must treat IPs as independent devices and utilize code running on the host CPU (i.e., CPU-centric) to manage collaboration between different IPs, resulting in significant overhead in both control and data exchange. Moreover, with the integration of many somewhat general-purpose IPs, domain-specific programming frameworks may be unable to utilize available IPs from other domains to perform the same functionality. For example, using DLA requires explicit programming in NivdiaTegra Xavier.
[0005] However, current research rarely investigates the programming productivity issues caused by increasing hardware heterogeneity; most studies remain focused on improving the performance and energy efficiency of individual IPs. Some work aims to improve SoC performance by scheduling IPs chained for streaming applications in certain scenarios or by adding shortcuts in the hardware. Others have proposed a fractal approach to address programming productivity, albeit on machine learning accelerators of varying scales. As a result, the increasing hardware heterogeneity is fundamentally changing the paradigm for building future SoC systems and raises critical questions about how to construct SoC systems with high productivity and high hardware utilization. Summary of the Invention
[0006] To at least partially address one or more of the technical problems mentioned in the background section, this disclosure provides solutions from several aspects. Firstly, it offers a new, unified system-on-a-chip (SoC) architecture framework (which may be referred to as a System-on-a-Chip Processor, or SaaP for short) that eliminates hardware heterogeneity from a software perspective, thereby improving programming productivity and hardware utilization. Secondly, it provides an architecturally flexible hybrid-scale instruction set to support new components for high productivity and SaaP, including memory vesicles for on-chip management and on-chip interconnects for data paths, thus building an efficient SaaP architecture. Thirdly, it provides a compilation method for compiling program code from various high-level programming languages into hybrid-scale instructions. Other aspects of this disclosure also provide solutions for branch prediction, exceptions, and interrupts within the instructions.
[0007] In a first aspect, this disclosure discloses an instruction processing apparatus, comprising: an instruction decoder for decoding mixed-scale (MS) instructions, the MS instructions including sub-instruction fields indicating one or more execution unit-specific sub-instruction information capable of executing the MS instructions; and an instruction dispatcher for dispatching the MS instructions to corresponding execution units according to the sub-instruction fields.
[0008] In a second aspect, this disclosure discloses an instruction execution method, comprising: decoding a mixed-scale (MS) instruction, the MS instruction including a sub-instruction field indicating sub-instruction information specific to one or more execution units capable of executing the MS instruction; and dispatching the MS instruction to a corresponding execution unit according to the sub-instruction field.
[0009] In a third aspect, this disclosure discloses a system-on-a-chip (SoC) including the instruction processing apparatus of the first aspect and a plurality of heterogeneous IP cores, the plurality of heterogeneous IP cores serving as the execution unit.
[0010] In the fourth aspect, this disclosure discloses a board that includes the system-on-a-chip mentioned in the third aspect above.
[0011] Based on the instruction processing device, instruction execution method, on-chip system, and board provided above, a new MS instruction set is provided to create a unified abstraction at the hardware and software interface. This hides the heterogeneity between different hardware or different instructions, ensuring that the hardware layer sees a unified MS instruction format. These MS instructions can then be distributed to different execution units for actual execution. Attached Figure Description
[0012] The above and other objects, features, and advantages of exemplary embodiments of this disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:
[0013] Figure 1 An example is shown of a typical SoC architecture;
[0014] Figure 2 This demonstrates the hardware heterogeneity on the SoC;
[0015] Figure 3 A typical timeline for a traditional SoC is shown;
[0016] Figure 4a A simplified diagram schematically illustrates the SaaP architecture according to an embodiment of this disclosure;
[0017] Figure 4b A traditional SoC architecture is shown for comparison;
[0018] Figure 5 The overall architecture of SaaP according to an embodiment of this disclosure is shown;
[0019] Figure 6 An example process for performing a task on a MISC architecture is shown below;
[0020] Figure 7 An exemplary flowchart of an instruction execution method according to an embodiment of this disclosure is shown;
[0021] Figure 8 An exemplary flowchart of an instruction execution method for a branch instruction according to an embodiment of this disclosure is shown;
[0022] Figure 9 An exemplary flowchart of an instruction execution method according to an embodiment of this disclosure is shown;
[0023] Figure 10 An example of instruction execution according to an embodiment of this disclosure is shown;
[0024] Figure 11 Several different data path designs are shown;
[0025] Figure 12 An exemplary flowchart of a compilation method according to an embodiment of this disclosure is shown;
[0026] Figure 13 An example program is shown;
[0027] Figure 14 A schematic diagram of the structure of a board card according to an embodiment of this disclosure is shown. Detailed Implementation
[0028] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.
[0029] It should be understood that the terms "first," "second," "third," and "fourth," etc., that may appear in the claims, specification, and drawings of this disclosure are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.
[0030] It should also be understood that the terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.
[0031] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection."
[0032] A System-on-a-Chip (SoC) is an integrated circuit chip that integrates all the key components of a system onto a single chip. SoCs are the most common integration solution in today's mobile / edge scenarios. Compared to motherboard-based solutions, their high integration improves system performance, reduces overall power consumption, and offers significantly smaller area costs.
[0033] Figure 1 An example is shown of a typical SoC architecture.
[0034] Due to performance requirements within limited area / power budgets, SoCs typically integrate numerous dedicated hardware IPs, often domain-specific architectures designed for computational purposes, particularly to accelerate domain-specific or application-specific applications. Some of these hardware IPs are custom-designed by the SoC designer, such as neural network processing IPs (the Neural Engine (NE) in Apple's A15, the Deep Learning Accelerator (DLA) in NVIDIA's Jetson Xavier, the Neural Processing Unit (NPU) in HiSilicon Kirin, and Samsung Exynos), while others are standardized by IP vendors, such as CPUs and GPUs from Arm or Imagination, DSPs from Synopsys or Cadence, FPGAs from Intel or Xilinx, and so on.
[0035] exist Figure 1 The example shows CPU 101, GPU 102, NPU 103, on-chip RAM 104, DRAM controller 105, arbiter 106, decoder 107, external storage interface 108, bus bridge 109, UART 110, GPIO 111, ROM interface 112, etc.
[0036] Traditional SoC designs utilize shared data buses or on-chip networks (NoC) to link components together. A common bus used for SoC on-chip interconnects is ARM's open standard Advanced Microcontroller Bus Architecture (AMBA).
[0037] exist Figure 1 In the example, the SoC uses shared buses to connect and manage the various functional blocks within the SoC. These shared buses include the Advanced High-Performance Bus (AHB) for high-speed connections and the Advanced Peripheral Bus (APB) for low-bandwidth, low-speed connections. Other network class topologies, also known as NoC, can be introduced, utilizing router-based packet-switched networks to manage even more components.
[0038] Integrating multiple different IPs leads to hardware heterogeneity on a SoC. Hardware heterogeneity includes heterogeneity of IPs within a SoC and heterogeneity of IPs between SoCs.
[0039] Figure 2 This illustrates the hardware heterogeneity on the SoC. The figure shows several IPs integrated on the SoC. For example, model A integrates a CPU and GPU on the SoC; model B integrates a CPU, GPU, and a neural engine (NE) for neural network processing on the SoC; model C integrates a CPU, GPU, and a neural processing unit (NPU) for neural network processing on the SoC; and model D integrates a CPU, GPU, a deep learning accelerator (DLA) for deep learning, and a programmable vision accelerator (PVA) on the SoC.
[0040] As the diagram shows, the IPs on the same SoC are different, for example, for different purposes. This heterogeneity of IPs within a SoC is due to the increasing integration of different types of IPs (especially those for computing purposes) into the SoC to achieve higher efficiency. New IPs are continuously being introduced into SoCs. For example, a new type of neural network processing IP has been widely introduced into recent mobile SoCs. Furthermore, the number of processing units in a SoC is also continuously increasing. For example, SoC model A mainly includes 10 processing units (2 large cores, 2 small cores, and a 6-core GPU); while in model B, the number of processing units increases to 30 (2 large general-purpose cores, 4 small general-purpose cores, a 16-core neural engine, and a 5-core GPU).
[0041] Regarding the heterogeneity of IP across SoCs, IPs implementing the same function on different SoCs can differ significantly because, for commercial reasons, companies always prefer to use their own proprietary IPs. For example, ... Figure 2 As shown in (b), (c), and (d), the same functionality (e.g., neural network processing) refers to different IPs. In a certain model B SoC, it is a Neural Engine (NE); in a certain model D, it is a Deep Learning Accelerator (DLA); and in a certain model C, it is a Neural Processing Unit (NPU). Furthermore, many computationally oriented IPs have a degree of generality for a specific domain (e.g., deep learning) or certain types of operations (e.g., GPUs with tensor operations).
[0042] Programming IP such as GPUs and NPUs for computational purposes can be achieved based on support from programming frameworks and vendors. For example, to accelerate neural network processing, application developers can directly use deep learning programming frameworks such as PyTorch, TensorFlow, and MXNet instead of manually managing them. These frameworks provide high-level programming interfaces (C++ / Python) to customize the IP, which is then implemented using the IP vendor's low-level interfaces. IP vendors make their hardware drivers compatible with these frameworks by providing different programming interfaces, such as PTX, CUDA, cuDNN, and NCCL.
[0043] However, programming frameworks require an immense amount of development work because they are required to bridge the gap between software and hardware diversity. Programming frameworks provide application developers with high-level interfaces to improve programming productivity, and these interfaces are carefully implemented to enhance hardware performance and efficiency. For example, TensorFlow, initially developed by approximately 100 developers, is now maintained by over 3000 contributors to support dozens of SoC platforms. Optimizing a single operator on a particular IP for thousands of TensorFlow operators can take a skilled developer several months. Even with a programming framework, different SoCs may require application developers to use different implementations. For instance, a program written for a specific SoC (e.g., model D) cannot run directly on the server-side DGX-1 of the GPU's TensorCores.
[0044] The difficulty in achieving high efficiency with programming frameworks stems from the fact that SoCs are managed by the host CPU. Since the programming framework, running on the host CPU, controls the entire execution process, control and data interaction are unavoidable. It handles control using CPU-IP-only interaction and data exchange using memory-IP-only interaction.
[0045] Figure 3 The diagram illustrates a typical timeline for a traditional SoC. As shown, the host CPU runs the programming framework for runtime management, where each call to the IP is initiated / terminated by the host CPU, resulting in significant runtime overhead. Data is stored in off-chip main memory, and the IP reads / writes data from main memory, incurring additional data accesses. For example, when running the YOLO neural network on a certain model D, control returns to the programming framework from the GPU 39 times, occupying 56.75MB of DRAM space, of which 95.06% is unnecessary. According to Amdahl's Law, the system's efficiency is finite, especially for programs composed of fragmented operations.
[0046] Inventive concept
[0047] Considering that exposing hardware heterogeneity to management software can lead to low productivity and low hardware utilization, this disclosure proposes a solution that allows the SoC hardware to manage heterogeneity itself. The inventors note that in classic CPUs, heterogeneous arithmetic logic units (ALUs) and floating-point units (FPUs) are treated as execution units in a pipeline and managed by hardware. Inspired by this, IP can intuitively also be viewed as execution units in an IP-level pipeline, i.e., a unified system-on-a-chip (SoC-as-a-Processor, SaaP).
[0048] Figure 4aA simplified diagram schematically illustrates the SaaP architecture according to an embodiment of this disclosure. In contrast, Figure 4b This illustrates a traditional SoC architecture, where single lines represent control flow and wide lines represent data flow.
[0049] like Figure 4a As shown, the SaaP in this embodiment refactors the SoC into a processor, including a system controller 410 (equivalent to a controller in the processor, i.e., a pipeline manager) for managing the hardware pipeline, including fetching instructions from system memory (e.g., DRAM 440 in the figure), decoding instructions, dispatching instructions, canceling instructions, and committing instructions. Multiple heterogeneous IP cores, including CPU cores, are integrated into the SoC as execution units in the hardware pipeline 420 (equivalent to arithmetic units in the processor) to execute instructions dispatched by the system controller 410. Therefore, SaaP can utilize hardware pipelines instead of a programming framework to manage heterogeneous IP cores.
[0050] Similar to the multiscalar paradigm, programs are divided into tasks, which can range from a single scalar instruction to an entire program. A task can be implemented on various types of IP cores and is dispatched to a specific IP core during execution. These tasks are called instructions in SaaP. Because tasks come in different sizes, this disclosure proposes a mixed-scale (MS) instruction to work with SaaP that has IP-level pipelines. The MS instruction is a unified instruction that can be applied to various heterogeneous IP cores. Therefore, hardware heterogeneity is transparent under the MS instruction. The MS instruction is fetched, decoded, dispatched, canceled, and committed by the system controller 410. The use of the MS instruction can fully utilize mixed-level parallelism.
[0051] Furthermore, an on-chip memory 430, such as on-chip SRAM or registers, can be provided for the SaaP to cache data related to the execution of the execution unit (IP core), such as input and output data. Thus, after data is moved from system memory to the on-chip memory, the IP core can interact with the on-chip memory for data access. The on-chip memory 430 is similar to a register in a processor; therefore, on-chip IP collaboration can also be implicitly implemented in a manner similar to register forwarding in a multi-scalar pipeline.
[0052] In the SaaP hardware pipeline, high hardware performance can be achieved by leveraging mixed-level parallelism using MS instructions and utilizing on-chip memory for data exchange between IP cores. Furthermore, SaaP allows any type of IP core to be integrated as an execution unit, and high-level code from application developers can be compiled for new IP cores with only minor modifications, thereby improving programming productivity.
[0053] In comparison, Figure 4b The traditional SoC shown is CPU-centric, with a programming framework running on the host CPU. Various IP cores are attached to the system bus as isolated devices and managed by software running on the host CPU. As can be seen from the diagram, in a traditional SoC, for control flow, there is only CPU-IP interaction; for data flow, there is only system memory (DRAM)-IP interaction.
[0054] In SaaP, SoCs are built using IP-level pipelines, with IP cores managed as execution units. This allows control flow to be managed by the pipeline manager, eliminating the need for a programming framework at runtime. Furthermore, leveraging a pipeline-like forwarding mechanism, data exchange can occur directly between different IP cores.
[0055] Extending CPU scalar pipelines to IP-level pipelines inevitably presents numerous challenges. One challenge lies in consistency. Because heterogeneous IP cores, such as deep learning accelerators, access data in blocks of varying sizes (e.g., tensors and vectors) rather than as scalar data, checking data dependencies and maintaining data consistency becomes extremely complex as data blocks flow concurrently through the pipeline. Consequently, register files, cache hierarchies, and data paths all require fundamental redesign. Another challenge is scalability. According to Amdahl's Law, the overhead of IP coordination (typically on the μs level) inadvertently limits the scalability of traditional SoCs. This overhead also prevents sub-μs-level cores from utilizing the IP, as this overhead can outweigh the execution time. Moreover, for scalability, SaaP should not favor time / area-intensive designs such as chained squashing and crossbar interconnects.
[0056] Despite the challenges from multiple sources, the inventors discovered that the root of the problem lies simply in the unclear ownership of shared data in traditional design concepts. In traditional SoCs, data can be accessed and modified by different IP cores at any time, and multiple copies of the data can exist. Therefore, to ensure proper program execution, complex mechanisms with significant overhead, such as bus snooping, atomic operations, transactional memory, and address resolution buffers, are required to maintain data consistency and IP collaboration consistency.
[0057] To avoid drawbacks caused by unclear ownership of shared data, SaaP SoCs are designed following the principle of Pure Exclusive Ownership (PXO). The principle is that data-related resources in the system, including on-chip buffers, data paths, data caches, memory, and I / O devices, are exclusively owned by a single IP core at any given time. The SaaP architecture and related designs provided in this disclosure embodiment will be described in detail below with reference to the accompanying drawings.
[0058] SaaP Overall Architecture
[0059] Figure 5 The overall architecture of SaaP according to embodiments of this disclosure is shown in more detail. Similar to the Tomasulo pipeline, SaaP may include an out-of-order five-stage pipeline.
[0060] As shown in the figure, in SaaP, the system controller, acting as a pipeline manager, can include multiple functional components to implement different functions during pipeline management. For example, instruction decoder 511 can decode the MS instructions proposed in this disclosure embodiment. Instruction dispatcher 512 can dispatch MS instructions. Instruction exit circuit 513 is used to complete instruction submission and exit completed MS instructions sequentially. MS instruction buffer 514 is used to buffer MS instructions. Renaming circuit 515 is used to rename the storage elements involved in the instruction to, for example, resolve potential data hazards. The system controller can utilize the renaming mechanism to implement any or more of the following processes: resolving data hazards on storage elements; MS instruction cancellation, MS instruction submission, etc. Exception handling circuit 516 is used to perform corresponding processing in response to exceptions thrown by the IP core. The functions of each component will be elaborated in the description of the relevant sections below.
[0061] Integrated heterogeneous IP cores (exemplary examples of CPU cores, GPU cores, DLA cores, etc. are shown in the figure) act as execution units to perform actual computations. These heterogeneous IP cores and related components (such as reservation station 521, IP instruction cache 522, etc.) can be collectively referred to as IP core complex 520.
[0062] SaaP also provides on-chip memory. In some implementations, on-chip memory can be implemented as a set of temporary registers (also called a set of memory bubbles) for buffering input and output data. Memory bubbles act as registers in the processor. A memory bubble can include multiple temporary registers of varying sizes to cache data related to the execution of multiple heterogeneous IP cores. For example, the size of a memory bubble can range from 64B, 128B, 256B, ... 256KB up to 512KB. Preferably, there are more small-capacity memory bubbles than large-capacity memory bubbles, thus better supporting task requirements of different scales. This set of memory bubbles can be collectively referred to as the memory bubble complex 530.
[0063] Between the storage bubble complex 530 and the IP core complex 520, an on-chip interconnect 540 is provided to provide a non-blocking data path connection between multiple heterogeneous IP cores and a set of storage bubbles. The on-chip interconnect acts as a shared data bus. In some embodiments, the on-chip interconnect 540 can be implemented based on a sorting network, thereby providing a non-blocking data path with minimal hardware cost and acceptable latency. In this document, the on-chip interconnect 540 may also be referred to as Golgi.
[0064] As mentioned earlier, SaaP SoCs are designed following the principle of Pure Exclusive Ownership (PXO) architecture. Therefore, in some embodiments, one of the aforementioned heterogeneous IP cores can be designated as the master core, responsible for managing the entire system. For example, the master core exclusively manages data exchange between system memory and storage bubbles. The master core also exclusively manages I / O operations with external devices. The master core can also control the operating system (OS) and runtime, and is responsible for at least one or more of the following processes: process management, page management, exception handling, and interrupt handling. For example, in branching and predictive execution, branching and predictive execution are implemented through exception handling, where impossible branches are treated as Impossible Branch Exceptions (UBEs). Static prediction can be used to implement branching and predictive execution. Considering the role and function of the master core, a CPU core with general-purpose processing capabilities is typically identified as the master core. In some embodiments, it is preferable to enhance the I / O capabilities of the master core, for example, by introducing a DMA module to alleviate the pressure of continuous data copying.
[0065] In some embodiments, non-parent IP cores can be divided into different IP lanes based on their function and / or type. The parent core itself belongs to a separate IP lane. Figure 5 The diagram illustrates the mother core lane, CPU lane, CPU lane, DLA lane, and so on. Subsequently, when scheduling MS commands, the MS commands can be assigned to appropriate IP lanes, at least in part, based on the task type of the MS command.
[0066] In general, SaaP utilizes MS instructions to execute the entire program. Initially, when the system controller retrieves an MS instruction, it decodes it to prepare data for execution. Data is loaded from system memory into a storage bubble or rapidly forwarded from another storage bubble. If there are no conflicts, the MS instruction is sent to the MS instruction dispatcher and then issued to the appropriate IP core (e.g., a DLA core) for actual execution. This IP core loads pre-compiled actual IP-specific code (e.g., DLA instructions) based on the issued MS instruction. The IP core then executes this actual code, much like execution on a conventional accelerator. After execution, the MS instruction exits the pipeline and submits its results.
[0067] The overall architecture and task execution process of the SaaP SoC in this application embodiment have been summarized above. The specific implementation of each part will be described in detail below. It is understood that although the implementation of each part is described within the SaaP SoC environment, these parts can also be used outside of the SaaP SoC in other similar environments, such as non-heterogeneous systems. This disclosure embodiment does not limit this in this respect.
[0068] MS (Mixed Scale) instruction
[0069] Hardware heterogeneity manifests in the hardware-software interface as differences in instruction formats, and the number of execution cycles per instruction varies significantly. Table 1 shows a comparison between different instruction sets. For scalar systems, there are typically two instruction sets: CISC (Complex Instruction Set Computer) and RISC (Reduced Instruction Set Computer). As shown in the table, the length of each CISC instruction is uncertain; some instructions are complex and have a high number of cycles, while others are simple and have a low number of cycles. Depending on the complexity of a single instruction, the number of instruction cycles ranges from 2 to 15 cycles. RISC instructions have a fixed length, and the number of instruction cycles per instruction is more uniform, approximately 1 to 1.5 cycles.
[0070] Due to the heterogeneity of SaaP SoCs, the instruction sets required by various IP cores (including CPUs and various xPUs) differ, such as in scale or granularity. To hide this heterogeneity (i.e., instruction format, execution cycle count, etc.), some embodiments disclosed herein provide a Mixed-scale (MISC) instruction set, similar in form to RISC, suitable for various IP cores. These IP cores (primarily various accelerators for computing purposes) mostly need to efficiently handle complex, large-granularity tasks; therefore, the execution cycle count (CPI) of a single MS instruction is longer than that of RISC, ranging from 10 to 10,000+ cycles, which is a relatively large range. Table 1 also shows the MISC provided in the embodiments disclosed herein.
[0071]
[0072] Table 1 Comparison of different instruction sets
[0073] Each instance of SaaP is a Mixed-Scale Instruction Set Computer (MISC). The MISC instruction set consists of MS instructions. Unlike RISC and CISC, MISC has its own unique design style.
[0074] First, MS instructions have mixed payload sizes, ranging from relatively small payloads (e.g., requiring only 10 cycles) to relatively large payloads (e.g., requiring over 10,000 cycles). Therefore, the payload carried by each MS instruction may require containers of different sizes to facilitate data retrieval and storage of calculated results. In this disclosed embodiment, a set of storage bubbles of varying sizes (e.g., from 64B to 512KB) mentioned above are used to store the input and / or output data required by the MS instructions, thereby supporting this mixed payload size of MS instructions.
[0075] Secondly, MS instructions are IP-independent, meaning they are unaware of the IP core. Specifically, instructions specific to each IP core (i.e., heterogeneous instructions) are encapsulated within MS instructions, and the format of the encapsulated MS instructions is independent of which specific IP core they are encapsulated in.
[0076] In some embodiments, an MS instruction may include a subinstruction field indicating one or more IP core-specific subinstruction information capable of executing the MS instruction. It is understood that the MS instruction will need to run on a specific IP core, meaning there is a piece of code that the IP core can recognize (i.e., IP core-specific code). This code also consists of one or more IP core-specific instructions, which are encapsulated within the MS instruction and are therefore called subinstructions. Thus, the system controller can dispatch the MS instruction to the corresponding IP core based on the subinstruction field. The subinstruction information may include the type of the subinstruction (i.e., the type of IP core or IP lane) and / or the address of the subinstruction. Various implementations can be used to represent the subinstruction information.
[0077] In one implementation, the addresses of one or more IP core-specific subinstructions can be placed in the subinstruction field. This method directly determines the subinstruction type and address in the MS instruction. However, in this implementation, since the same MS instruction may run on multiple heterogeneous IP cores, the length of the MS instruction will vary depending on the number of IP core types capable of executing that MS instruction.
[0078] In another implementation, a bit sequence can be used to indicate whether the MS instruction contains a corresponding type of subinstruction, and a starting address can be used to represent the address of the first subinstruction segment. The length of the bit sequence can be the number of IP core types or IP lane types, so that each bit in the bit sequence can be used to indicate whether a corresponding type of subinstruction exists. The address of the first subinstruction segment is obtained directly from the starting address, and the addresses of subsequent IP lanes corresponding to subinstructions can be indexed in a fixed way (e.g., at address intervals) or implemented by directly jumping to the MS instruction. This disclosed embodiment does not limit the specific format implementation of the MS instruction.
[0079] MS instructions are defined to perform complex functions. Therefore, each MS instruction performs a complex function, such as convolution, which is broken down into fine-grained IP-specific code (i.e., sub-instructions) for actual execution, such as RISC instructions. The IP-specific code can be compiled from a standard library (e.g., `std::inner_product` from libstdc++ for inner product) or generated from a vendor-specific library (e.g., `cublasSdot` from cuBLAS, also for inner product operations). This makes it possible for SaaP to integrate different types of IP, as the same MS instruction can be flexibly issued to different types of IP cores. Therefore, heterogeneity is hidden from application developers, which also increases the robustness of SaaP.
[0080] As can be seen from the above, regardless of which IP core the subinstruction is used for, such as CPU, GPU, DLA, NPU, etc., the format of the MS instruction will not change. Therefore, from this perspective, the MS instruction is IP-independent.
[0081] Secondly, MS instructions have a finite number of atomities. For data management, each MS instruction accesses a maximum of three memory bubbles: two source memory bubbles and one destination memory bubble. That is, for data management, each MS instruction has at most two input data fields and one output data field, used to indicate the data information associated with executing the MS instruction. In some implementations, these data fields can be represented by the numbers of the associated memory bubbles, for example, indicating two input memory bubbles and one output memory bubble respectively. The finite number of atomities reduces the complexity of conflict resolution, renaming, data path design, and compiler toolchains. For example, if the number of atomities of MS instructions were unlimited, the decoding time differences between different MS instructions would be very large, leading to hardware pipeline irregularities and inefficiencies. For functions or features with a high number of atomities (e.g., more than 3 atomities), currying can be used. Currying is a technique that converts multivariate functions into sequences of single-variable functions, such as through nesting, chaining, etc. This allows functions / features with any number of inputs and outputs to be converted into functions / features that satisfy the finite number of atomities of MS instructions.
[0082] Finally, MS instructions have no side effects. "No side effects" here means that the execution state of the current instruction does not affect the execution of subsequent instructions. In other words, if the current instruction needs to be revoked, it can be revoked without its residual state affecting subsequent instructions. Aside from modifying data in the output memory bubble, the execution of MS instructions leaves no observable side effects on the SaaP architecture. The only exception is MS instructions executed on the parent core, because the parent core can operate on system memory and external devices. This constraint is crucial for implementing Hybrid Parallelism (MLP) because it allows for easy rollback when, for example, an MS instruction needs to be revoked based on predictive execution requirements. In other words, the data field of an MS instruction executed on a non-parent core IP core can only point to a memory bubble, not system memory. Furthermore, the memory bubble corresponding to the output data is exclusively assigned to the IP core executing the MS instruction.
[0083] Therefore, by providing a new MS instruction set and creating a unified abstraction at the hardware and software interface, the heterogeneity between different hardware or instructions can be hidden, resulting in a unified MS instruction format visible at the hardware level. These MS instructions can then be distributed to different IP cores for actual execution.
[0084] Figure 6This illustration demonstrates an example process of executing a task on a MISC architecture to better understand the implementation of MS instructions. The illustrated MISC architecture, for example, has a mother core and an IP core. The task to be executed is to make sandwiches (materials: bread and meat) and vegetable salads (materials: vegetables). For ease of illustration, [details omitted]. Figure 6 In this context, bread is named A, meat is named B, vegetables are named C, sandwiches are named D, and salads are named E. The mother kernel manages system memory; therefore, the mother kernel first loads the materials to be processed from system memory into storage bubbles. Then, the IP cores can process the materials in the storage bubbles. The above task can be represented by the following MS instruction stream:
[0085] 1) "Load Bread" v1,void,void
[0086] 2) "Load Meat" v2, void, void
[0087] 3) "Make Sandwich" v1, v1, v2
[0088] 4) "Store Sandwich"void,v1,void
[0089] 5) "Load Green" v1,void,void
[0090] 6) "Make Salad" v1, v1, void
[0091] 7) "Store Salad"void,v1,void
[0092] It is understandable that MS instructions, when executed on both the parent and IP cores, should provide core-specific code forms, i.e., core-specific sub-instructions, so that each core knows how to execute the corresponding task. For simplicity, these sub-instructions are only shown in the MS instruction stream above, without distinguishing between different forms. The memory bubbles (v1, v2) used in MS instructions are logical numbers. In actual execution, memory bubbles are renamed to different physical numbers to resolve WAW (write-after-write) dependencies and support out-of-order predictive execution. The void in the instruction indicates that the corresponding field does not require a memory bubble, such as when system memory is involved.
[0093] exist Figure 6In the diagram, ① represents the initial state; ② represents the mother kernel executing the "Load Bread" MS instruction. The Load instruction involves accessing system memory, therefore it is assigned to the mother kernel for execution. The mother kernel retrieves data from system memory and stores it in the storage bubble v1. The specific memory access address information can be placed in an additional instruction field; this disclosed embodiment does not impose any limitations in this regard. ③ represents the mother kernel executing the "Load Meat" instruction. Similar to the "Load Bread" instruction, the mother kernel retrieves data from system memory and stores it in the storage bubble v2.
[0094] Next, step ④ is to execute the "Make Sandwich" instruction. This MS instruction is dispatched to the IP core for processing because it requires a significant amount of processing time. According to the original instruction, the IP core needs to retrieve bread from v1 and meat from v2, prepare the sandwich, and then place it into v1. Here, since the v1 to be written to and the v1 to be read from are the same, there is a read-after-write (RAW) dependency, meaning that the data in v1 must be completely read before it can be written. However, this method is impractical because the MS instruction can be very large, for example, requiring tens of thousands of cycles, and the sandwiches prepared in between need to be stored. To resolve this data hazard, a storage bubble renaming mechanism can be used. For example, before dispatching the MS instruction, the storage bubble renaming mechanism can be used... Figure 5 The storage bubble renaming circuit 515 shown renames the logical name of the storage bubble in the MS instruction to a physical name, thereby eliminating data hazards. Simultaneously, the storage bubble renaming circuit 515 maintains the mapping relationship between the physical name and the logical name. Figure 6 In the example, the storage bubble v1 corresponding to the output data of the “MakeSandwich” instruction is renamed to storage bubble v3, so the made sandwich is put into v3. Figure 6 The ellipsis in v3 indicates that the write process will take some time and will not be completed quickly.
[0095] In step ⑤, because making the sandwich takes a considerable amount of time, the subsequent "StoreSandwich" instruction cannot be executed yet. However, the following "Load Green" instruction has no dependency on the preceding instructions, so it can be executed in parallel. Similarly, the storage bubble v1 involved in the "Load Green" instruction also involves a read-after-write relationship. Therefore, a storage bubble renaming mechanism can be used to rename and map the corresponding storage bubble v1 to storage bubble v4. Likewise, the parent kernel executes the "Load Green" instruction to retrieve data from system memory and write it into the v4 storage bubble.
[0096] In step ⑥, since the IP core is already occupied for making sandwiches, to improve efficiency, according to the scheduling policy, the "MakeSalad" instruction can be assigned to the currently idle mother core for execution. The status of each core can be marked, for example, by a bit sequence to facilitate instruction dispatching by the instruction dispatcher. Similarly, a renaming mechanism is also applied here. The mother core takes vegetables from storage bubble v4, makes a salad, and puts it into storage bubble v5.
[0097] In step ⑦, once the sandwich is made, the previously blocked "Store Sandwich" instruction can be executed. The Store instruction involves accessing system memory, so it is dispatched to the main kernel for execution. The main kernel retrieves the data from the stored bubble v3 and stores it in system memory.
[0098] In step ⑧, once the salad is prepared, the "Store Salad" command can be executed. The kernel will then retrieve the data from the stored bubble v5 and store it in system memory.
[0099] It is important to note that in steps ⑦ and ⑧, even if the salad is prepared before the sandwich, the "Store Salad" command must be executed after the "Store Sandwich" command to ensure sequential submission, thus preventing any side effects in the event of command cancellation.
[0100] As can be seen from the example process above, once the data is ready, the IP core can begin processing. "MakeSandwich" takes a considerable amount of time; therefore, "MakeSandwich" is executed on the parent core and completed ahead of time, thus fully leveraging hybrid level parallelism (MLP). Consequently, the execution of different IP cores does not interfere with each other; that is, they can be executed out of order, but are committed sequentially.
[0101] System Controller
[0102] The processing of MS instructions, both between and within the instructions themselves, is uniformly managed by the system controller (also known as the instruction processing unit). The functions of each component within the system controller are described in detail below. SaaP SoC employs out-of-order pipelines to exploit mixed-level parallelism between IP cores. The pipeline can contain five stages: fetch & decode, conflict resolution, dispatch, execution, and exit.
[0103] Figure 7 An exemplary flowchart of an instruction execution method according to an embodiment of this disclosure is shown. The following description may also be referenced. Figure 5 The SaaP architecture shown is used for understanding. Furthermore, for ease of description and understanding, Figure 7The instruction execution process, which includes a complete pipeline, is illustrated. However, those skilled in the art will understand that some steps may only occur under specific circumstances and are therefore not necessary in all cases. Their necessity can be determined based on the specific circumstances.
[0104] First, instruction fetching and decoding are performed in step 710. At this stage, the MS instruction is fetched from the MS instruction register 514 based on the MS program counter (PC), and the instruction decoder 511 decodes the fetched MS instruction to prepare operands. The decoded MS instruction can then be placed in the instruction queue of the instruction decoder 511.
[0105] As described above, an MS instruction includes a subinstruction field that indicates one or more IP core-specific subinstruction information that enables the execution of the MS instruction. This subinstruction information may, for example, indicate the type of the subinstruction (i.e., the type of IP core or IP lane) and / or the address of the subinstruction.
[0106] In some embodiments, when the MS instruction is retrieved and decoded, the corresponding sub-instruction can be retrieved in advance and stored in a specified location, such as the sub-instruction buffer 522, based on the decoding result. Figure 5 This is also known as the IP instruction buffer. Therefore, when the MS instruction is issued to the corresponding IP core for execution, the IP core can retrieve the corresponding sub-instruction from the sub-instruction buffer 522 for execution.
[0107] In some cases, the MS instruction may be a branch instruction. In this disclosed embodiment, static prediction is used to determine the direction of the branch instruction, that is, to determine the PC value of the next MS. The inventors analyzed the branching behavior in benchmark programs and found that 80.6% to 99.8% of branches in large-scale instruction branches can be correctly predicted at compile time. Since large-scale instructions determine the overall execution time, static prediction is used to perform branch prediction in this disclosed embodiment, thereby eliminating the need for a hardware branch predictor. Therefore, whenever a branch is encountered, it is always assumed that the next MS instruction is a possible branch direction predicted by static prediction.
[0108] When a branch is mispredicted, an unlikely branch exception (UBE) will be triggered. When an exception occurs, the erroneous MS instruction needs to be rolled back, the next MS instruction count is set to the unlikely branch of the UBE, or an exception trap occurs in other cases. The handling of branching and predictive execution will be described in detail later.
[0109] Next, the pipeline proceeds to step 720, where potential conflicts are resolved. At this stage, retrieved MS instructions are queued to resolve conflicts. Possible conflicts include (1) data hazards; (2) structural conflicts (e.g., no available space in the exit unit); and (3) exception violations (e.g., blocking an MS instruction that cannot be easily undone until it is confirmed to have been taken).
[0110] In some embodiments, a storage bubble renaming mechanism can be used to resolve data hazards such as read-after-write (WAR) and write-after-write (WAW). The storage bubble renaming circuit 515, when a data hazard exists on a storage bubble involved in an MS instruction, renames the logical name of the storage bubble to its physical name before dispatching the MS instruction, and maintains the mapping between the physical and logical names of the storage bubble. Through the storage bubble renaming mechanism, SaaP can support faster MS instruction rollback (achieved by simply discarding the renaming mapping of the output data storage bubble) and out-of-order execution without WAW hazards.
[0111] After resolving any potential conflicts, the pipeline proceeds to step 730, where the instruction dispatcher 512 dispatches the MS instruction.
[0112] As described above, the MS instruction has a sub-instruction field that indicates the IP core capable of executing the MS instruction. Therefore, the instruction dispatcher 512 can dispatch the MS instruction to the corresponding IP core based on the information in the sub-instruction field. Specifically, it first dispatches the instruction to the reserved station to which the IP core belongs for subsequent transmission to the appropriate IP core.
[0113] In some embodiments, IP cores can be divided into different IP lanes according to their function and / or type, with each lane corresponding to a specific IP core model. Correspondingly, reserved stations can also be grouped according to lanes, for example, each lane can correspond to one reserved station. Figure 5 The diagram illustrates the mother core lane, CPU lane, DLA lane, and so on. Different lanes are suitable for executing different types of tasks. Therefore, when scheduling and dispatching MS commands, the MS commands can be assigned to the appropriate reserved station for the appropriate lane, at least in part, based on the task type of the MS command, so that they can be subsequently launched to the appropriate IP core.
[0114] In some embodiments, in addition to considering task type, scheduling can be performed among multiple IP lanes capable of executing the MS instruction based on the processing status of each IP lane, thereby improving processing efficiency. Since the same MS instruction may have multiple different implementations executing on multiple IP cores, the processing pressure on bottleneck lanes can be alleviated by selecting the assigned lane according to an appropriate scheduling strategy. For example, MS instructions involving convolution operations can be assigned to either the GPU lane or the DLA lane. Both can be executed efficiently, and one can be selected based on the pressure of the two lanes to accelerate processing. Scheduling strategies can include various rules, such as selecting the IP core with the highest throughput or the IP core with the shortest number of sub-instructions, etc., and the embodiments disclosed herein are not limited in this respect.
[0115] In some embodiments, certain types of MS instructions must be dispatched to specific IP cores. For example, as mentioned earlier, among multiple heterogeneous IP cores, one IP core can be designated as the master core, responsible for managing the entire system. Therefore, some MS instructions involving system management must be dispatched to the master core for execution.
[0116] Specifically, the master core exclusively manages data exchange between system memory and storage bubbles. Therefore, memory access type MS instructions that access system memory are dispatched to the master core. The master core also exclusively manages I / O operations with external devices. Therefore, I / O type MS instructions, such as those for display output, are also dispatched to the master core. The master core can also control the operating system (OS) and runtime, and is responsible for at least one or more of the following: process management, page management, exception handling, and interrupt handling. Therefore, interrupt handling MS instructions are dispatched to the master core. Furthermore, when certain MS instructions cannot be processed by other IP cores, for example, because other IP cores are busy, they can be dispatched to the master core for processing. Additionally, depending on the MS instruction scheduling policy, some MS instructions may be allocated to the master core for processing. These are not listed here.
[0117] Next, the pipeline proceeds to step 740, where MS instructions can be executed out of order by the IP core.
[0118] Specifically, the IP cores assigned to instructions can utilize the actual IP-specific code to execute the functions of MS instructions. For example, based on the assigned instruction, the IP core retrieves the corresponding subinstruction from the subinstruction cache / IP instruction cache 522 and executes it. At this stage, the Tomasulo algorithm can be implemented to organize these IP cores, thereby supporting hybrid-level parallelism (MLP). Once the dependencies on the memory bubbles are resolved, MS instructions can be continuously assigned to the IP core complex, allowing for out-of-order execution of these instructions.
[0119] Note that in the SaaP SoC provided in this disclosure embodiment, the IP core is unaware of the SaaP architecture because intrusive modifications to the IP core are prohibited. To accommodate SaaP, the IP core is encapsulated using an adapter that directs access to the program to the IP instruction cache 522 and access to the data to the memory bubbles. The program can be an accelerator interface signal (e.g., a CSB control signal for DLA) or a piece of IP-specific code implementing MS instructions (e.g., for a programmable processor such as a CPU / GPU). Arithmetic MS instructions perform operations on data stored in a set of memory bubbles. These memory bubbles can be multiple registers of varying sizes. Each IP core has two data read ports and one data write port. During execution, the physical memory bubbles are exclusively connected to the ports, so from the IP core's perspective, the memory bubbles function like main memory in a traditional architecture.
[0120] Finally, the pipeline proceeds to step 750, the exit stage. At this stage, the MS instruction exits the pipeline and submits the result. Figure 5 The instruction exit circuit 513 is used to exit completed MS instructions sequentially. Upon exiting an MS instruction, the execution result is submitted by confirming the renaming mapping of the storage bubble corresponding to the MS instruction's output data. That is, submission is completed by permanently acknowledging the renaming mapping of the storage bubble for the output data in the renaming circuit 515. Since only the renaming mapping is acknowledged, no data is actually buffered or copied, thus avoiding the additional overhead of copying data when dealing with large amounts of data (common in various computation-oriented IP cores).
[0121] It should be understood that although the execution process of MS instructions is described in the SaaP SoC environment, the MS instruction set can also be applied to other environments, not limited to environments with heterogeneous IP cores. For example, it can also be used in homogeneous environments, as long as the execution unit of the MS instructions can independently parse and execute sub-instructions. Therefore, in the description above, the IP core can be directly replaced with the execution unit, and the parent core can be replaced with the main execution unit; the above method still applies.
[0122] Branching and Predictive Execution
[0123] Branch instructions may also appear in the MS instruction stream, causing control dependencies. These dependencies are actually related to the program counter (PC) of the MS instruction, and the PC value is used during instruction fetching. If branch instructions are not handled well, the fetching of the next instruction will be affected, leading to pipeline congestion and impacting pipeline efficiency. Therefore, it is necessary to provide effective branch prediction support for MS instructions, that is, to be effective for both large-scale and small-scale instructions.
[0124] In traditional CPU processing, branch conditions are calculated during decoding to determine the correct branch target, allowing the next instruction to be fetched from the address of the branch jump location during the next instruction fetch cycle. This branch condition calculation and setting the next PC value to the correct branch target typically only incurs a few clock cycles of overhead, which is very small and can be completely offset by the pipeline in a conventional CPU instruction pipeline. However, in a microprocessor (MS) instruction stream, if a branch MS instruction is mispredicted, it means that the misprediction is only discovered at some point in the entire execution of the MS instruction stream. This point in time could be hundreds, thousands, or even more clock cycles away from when the branch MS instruction began execution. Therefore, in an MS instruction pipeline, it is impossible to wait until the exact jump point is known before determining the PC value of the next MS instruction. This would result in a very large prediction overhead.
[0125] The inventors analyzed the branching behavior in five benchmark programs and found that 80.6% to 99.8% of branches in large-scale instruction branches could be correctly predicted at compile time, meaning they could be predicted statically. Since large-scale instructions account for a large portion of the total execution time and determine the overall execution time, this disclosed embodiment uses static prediction to perform branch prediction, thereby eliminating the need for a hardware branch predictor.
[0126] Figure 8 An exemplary flowchart of an instruction execution method for a branch instruction according to an embodiment of this disclosure is shown. The method is executed by a system controller.
[0127] As shown in the figure, in step 810, the MS instruction is decoded. The MS instruction has a variable cycle per instruction (CPI). As mentioned earlier, the CPI of the MS instruction can range from 10 cycles to over 10,000 cycles. This variable CPI characteristic of the MS instruction also makes it difficult to use dynamic prediction.
[0128] Next, in step 820, in response to the MS instruction being a branch instruction, the next MS instruction is obtained based on the branch indication information, which indicates possible branch targets and / or impossible branch targets.
[0129] Static prediction mechanisms can leverage compiler hints for static prediction. Specifically, during instruction compilation, branch indication information can be determined based on static branch prediction and inserted into the MS instruction stream.
[0130] Depending on the static branch prediction method, the branch indication information can contain different content. For example, static prediction always takes the possible branch target as the address of the next MS instruction. In some cases, to ensure the time locality of the instruction register, the possible branch target can usually be adjacent to the current MS instruction. Therefore, in these cases, the branch indication information may only need to indicate the impossible branch target. In other cases, the branch indication information may indicate both possible and impossible branch targets simultaneously. Therefore, when obtaining the next MS instruction based on the branch indication information, the possible branch target indicated by the branch indication information can be determined as the next MS instruction.
[0131] Since it involves prediction, errors are possible. When a branch prediction error occurs, all instructions following that branch instruction need to be canceled. The longer the pipeline stage, the more instructions need to be canceled due to branch prediction errors, and the greater the efficiency loss of the instruction pipeline. Because MS instructions use static prediction, the next MS instruction is fetched in a fixed manner before the branch condition is determined. These instructions can be executed out of order, but must be committed in the order described above. Therefore, when an error in the prediction direction of a branch instruction is found, it is necessary to revert to the correct next MS instruction. At this point, an exception mechanism is needed to correct the erroneous prediction.
[0132] Optionally or additionally, in step 830, when a prediction error occurs, the system controller receives an Impossible Branch Exception (UBE) event. This UBE event is triggered by an execution unit (e.g., an IP core) executing conditional computation instructions associated with the branch instruction. The UBE event indicates that, according to the conditional computation, the branch direction should be an impossible branch target, meaning that the previous branch prediction was incorrect.
[0133] At this point, in step 840, the system controller, in response to the UBE event, needs to perform a series of operations to resolve the branch prediction error. These operations include: undoing the MS instruction following the branch instruction; committing the MS instruction before the branch instruction; and determining the impossible branch target indicated by the branch indication information as the next MS instruction. This processing corresponds to a precise exception, that is, when an exception occurs, all instructions before the instruction interrupted by the exception are executed, while all instructions after the instruction are executed as if they were not executed. Since the UBE event is an exception caused by a branch prediction error, the instruction interrupted by the exception is the MS instruction of that branch.
[0134] Different operations can be performed to undo MS instructions depending on their state. MS instructions typically fall into three states: currently executing in an execution unit; completed execution; or yet to be executed. These different states can affect different hardware and software, so these effects need to be mitigated. For example, if the instruction is currently executing in an execution unit, the execution unit currently executing these undoable MS instructions needs to be terminated; if the instruction has written to registers (e.g., memory bubbles) during or after execution, these registers written to by the undoable MS instruction need to be discarded; if the instruction has not yet been executed, it only needs to be removed from the instruction queue. Of course, since the instruction queue records all uncommitted instructions, instructions in the currently executing or completed execution state also need to be removed from the instruction queue.
[0135] Therefore, in some embodiments, the MS instruction following the branch reversal instruction includes: canceling the reversed MS instructions in the instruction queue; terminating the execution unit executing the reversed MS instructions; and discarding the temporary registers written by the reversed MS instructions.
[0136] As described in the instruction exit process above, when an MS instruction exits, it submits the execution result by confirming the renaming mapping of the storage bubble corresponding to its output data. Therefore, when discarding registers written by these revoked MS instructions, it is only necessary to delete the corresponding mapping from the record storing the renaming mapping between the physical and logical names of these registers. As mentioned earlier, this storage bubble renaming mechanism supports faster MS instruction revocation, simply by discarding the renaming mapping of the output data storage bubble.
[0137] Therefore, in the MS instruction pipeline, handling branch MS instructions through static prediction can save hardware resources and adapt to the large CPI characteristics of MS instructions, thus improving pipeline efficiency. Furthermore, handling branch prediction errors through exception mechanisms can further save hardware resources and simplify processing.
[0138] Exception and interrupt handling
[0139] As can be seen from the preceding branch prediction processing, the cost of undoing a large number of MS instructions can be very high. Therefore, this disclosure proposes an instruction execution scheme that can block MS instructions that may cause high undo costs until all potentially discarded instructions before that instruction have been executed, i.e., the state has been determined. This instruction execution scheme can significantly improve the processing efficiency of the MS instruction pipeline in exception and interrupt handling.
[0140] Figure 9An exemplary flowchart of an instruction execution method according to an embodiment of this disclosure is shown. The method is executed by a system controller.
[0141] As shown in the figure, in step 910, when an MS command is transmitted, it is checked whether the MS command may be discarded.
[0142] In some embodiments, checking whether an MS instruction might be discarded includes checking whether the MS instruction has a potentially discardable tag. Potentially discardable tags can be inserted by the compiler at compile time based on the type of the MS instruction. For example, the compiler can insert a potentially discardable tag when it finds that the MS instruction is a conditional branch instruction or that other exceptions might occur.
[0143] Next, in step 920, when it is determined that the MS instruction may be discarded, the issuance of a specific MS instruction following the MS instruction is blocked.
[0144] Specific MS instructions can be large-scale MS instructions or MS instructions that typically have high undo costs. Specifically, specific MS instructions can be determined based on one or more of the following conditions: the size of the temporary register (memory bubble) corresponding to the output data of the MS instruction is greater than a set threshold; the MS instruction performs a write operation on system memory; the MS instruction's execution time exceeds a predetermined value; or the MS instruction is executed by a specific execution unit. When the size (capacity) of the memory bubble for the output data exceeds the set threshold, it indicates that the output data volume of the MS instruction is relatively large, and the corresponding undo cost is also high. Blocking MS instructions that write to system memory is mainly to ensure memory consistency.
[0145] After these specific MS instructions are blocked, the MS instructions preceding them will still be issued and executed normally. Depending on the possible outcomes of these normally issued and executed MS instructions, they can be handled individually.
[0146] On the one hand, in step 930, when all potentially discarded MS instructions that caused the blocking of that specific MS instruction have been executed normally, in response to this event, the blocking specific MS instruction can be issued for execution by the execution unit. It can be understood that at this point, it can be determined that the specific MS instruction will not be canceled due to the preceding instructions, and therefore the normal issuance and execution of the instruction pipeline can continue.
[0147] On the other hand, in step 940, when an exception occurs in the execution of any potentially discarded MS instruction that caused the blocking of that particular MS instruction, exception handling is performed in response to this exception event. Similarly, this exception handling corresponds to a precise exception, requiring the cancellation of the MS instruction that caused the exception and the MS instructions executed afterward, the submission of MS instructions preceding the MS instruction, and the MS instruction of the corresponding exception handler as the next MS instruction.
[0148] Similar to the description in branch prediction processing above, revoking the MS instruction that caused the exception and subsequent MS instructions includes: canceling these revoked MS instructions from the instruction queue; terminating the execution unit executing these revoked MS instructions; and discarding the registers written by these revoked MS instructions. Similarly, discarding the registers written by these revoked MS instructions includes deleting the corresponding mapping from the record storing the renaming mapping between the physical and logical names of these registers.
[0149] When the type of exception event is an impossible branch exception (UBE) event triggered by an MS instruction of the branch type in the aforementioned branch prediction processing, in addition to the exception handling described above, it is also necessary to determine the impossible branch target indicated by the branch indication information attached to the MS instruction as the next MS instruction after exception clearance. Thus, after the exception handling is completed, the instruction pipeline can normally jump to the correct branch direction and continue execution.
[0150] Figure 10 An example of instruction execution according to an embodiment of this disclosure is shown.
[0151] As shown in the figure, (a) illustrates the initial state of the MS instruction stream in the instruction queue, which includes 5 MS instructions to be executed. The #1 MS instruction carries a possible discard tag, and the different widths occupied by the instructions represent different scales. The #3 MS instruction is a large-scale MS instruction, and the rest are small-scale MS instructions. Different backgrounds of the instructions represent their different states, such as waiting, blocked, issued, executed, exited, exception, and canceled, as shown in the legend.
[0152] (b) illustrates the instruction issuance process. Small-scale instructions are issued as quickly as possible, while large-scale instructions are blocked by any previously issued instruction that may be discarded. Specifically, instruction #0 is issued first, followed by instruction #1. During the issuance of instruction #1, it is discovered that this instruction may be discarded. At this point, subsequent large-scale instructions are blocked. In this example, instruction #2, being small-scale, can still be issued normally; while instruction #3, being a large-scale instruction, is blocked, and subsequent instructions are also in a waiting state.
[0153] (c) illustrates the instruction execution process. In this example, instruction #2 may be executed first. Since the instructions preceding it have not yet been executed, it needs to wait to ensure sequential submission.
[0154] (d1)-(h1) show the processing procedure when no exception occurs during the execution of the aforementioned instruction; (d2)-(g2) show the processing procedure when an exception is thrown by the aforementioned instruction.
[0155] Specifically, (d1) shows that instruction #1 also completed normally without being discarded. At this point, the large-scale instruction #3, which was blocked due to instruction #1, can be issued, and the subsequent instruction #4 can also be issued normally. (e1) shows that instructions #0, #1, #2, and #4, being small in scale, have all completed execution, while instruction #3 is still in execution. (f1) shows that instructions #0, #1, and #2 are committed sequentially, while instruction #4 must wait for instruction #3 to complete before it can be committed. (g1) shows that instruction #3 has also completed execution. (h1) shows that instructions #3 and #4 are committed sequentially.
[0156] On the other hand, when an exception occurs during the execution of instruction #1, as shown in (d2), an exception handling procedure will be executed. The exception handling process typically includes exception preparation, determining the source of the exception, saving the execution state, handling the exception, restoring the execution state, and returning. For example, in... Figure 5 The exception handling circuit 516 shown can record whether an exception has occurred and adjust the address of the next MS instruction based on the processing result.
[0157] When handling exceptions, precise exception handling is performed. As shown in (e2) and (f2), the #0 instruction, which precedes the #1 instruction that triggers the exception, continues execution and completes its commit. The #2 instruction, issued after the #1 instruction that triggers the exception, although already executed, is also rolled back, as shown in (g2). At this point, #3 and #4, which have been blocked and not yet issued, are in a waiting state, thus avoiding the overhead of rollback.
[0158] If the exception triggered by instruction #1 is the UBE event described earlier, meaning instruction #1 is a branch instruction, then the impossible branch target indicated by the branch instruction can be determined as the next MS instruction after exception clearance, based on the branch indication information attached to the branch instruction. In other words, after exception handling, the pipeline will jump to the MS instruction corresponding to the impossible branch target.
[0159] If the exception is of another type, such as a denominator of zero in division, the process will jump to an exception handler, which may modify the denominator to a very small non-zero value. After the exception is handled, instruction #1 will be re-executed and normal instruction pipeline processing will continue.
[0160] In contrast to exceptions, interrupts originate from outside the SoC and are therefore unpredictable. However, SaaP does not need to precisely stop at the point where the interrupt signal is triggered. When an interrupt occurs, SaaP blocks all pending MS instructions and waits for all already issued MS instructions to complete and exit.
[0161] In SaaP, most system management exceptions, such as bad allocation, page faults, and segmentation faults, can only be raised by the parent kernel and are therefore caught and handled within the parent kernel. Other components and IP cores in the SaaP architecture are neither affected by these exceptions nor aware of them.
[0162] Storage bubble
[0163] In SaaP, storage vesicles are used as an alternative to registers for mixed-scale data access. Storage vesicles can be independent, mixed-size single-port scratchpads, with capacities ranging from 64 bytes to 512 KB. In SaaP, storage vesicles can function similarly to registers of mixed capacities for use by MS instructions. In this document, "storage vesicle complex" refers to a physical "register" file composed of storage vesicles, not a fixed-size register. Preferably, there are more small-capacity (e.g., 64 bytes) storage vesicles than large-capacity (e.g., 512 KB) storage vesicles to better match program requirements and support tasks of different scales. Physically, each storage vesicle can be a single SRAM or register with two read ports and one write port. These storage vesicles are designed to better match mixed-scale data access patterns and can serve as the basic unit of data management in SaaP.
[0164] Two IP cores cannot access the same memory bubble simultaneously. Therefore, data dependencies can still be managed as simply as with sequential scalar processors, and on-chip IP collaboration can be managed through a hardware MS instruction pipeline.
[0165] Data Path
[0166] To enable access to any memory bubble from any IP core, a complete connection between the IP core complex and the memory bubble complex is required. Common solutions include data buses (e.g., in CPUs) or cross-matrixes (e.g., in multi-core systems). However, neither of these connections meets the efficiency requirements. For example, data buses can lead to contention, and cross-matrixes are very space-consuming, even with only a few dozen cores. To achieve non-blocking data transfer at an acceptable cost, embodiments of this disclosure construct an on-chip interconnect data path based on a sorting network, referred to as Golgi.
[0167] Figure 11Several different data path designs are shown, including (a) a data bus, (b) a cross matrix, and (c) a Golgi diagram provided in the embodiments disclosed herein.
[0168] As can be seen from (a), the data bus cannot provide non-blocking access and requires a bus arbiter to resolve access conflicts. As can be seen from (b), the cross matrix can provide non-blocking access with lower latency, but it requires O(mn) switches, where m is the number of ports of the IP core and n is the number of ports storing the bubbles.
[0169] In Golgi's diagram (c), the connectivity problem is viewed as a Top-K sorting network, where storage bubble ports are sorted based on destination IP port numbers. The on-chip interconnect comprises a bitone sorting network consisting of multiple comparators and switches. When m IP core ports need to access n storage bubble ports, this bitone sorting network is used to sort the relevant storage bubble ports based on the destination IP core port index, thus establishing a data path between the m IP core ports and the n storage bubble ports.
[0170] For the example in (c), when it is necessary to map storage bubbles {a,c,d} to IP cores {#3,#1,#2} respectively, Gorky considers the mapping as an ordering of all storage bubbles {a,b,c,d} with values {#3,#+∞,#1,#2} respectively, where unused ports are assigned destination numbers +∞.
[0171] Specifically, as shown in (c), starting from the storage bubble {a,b,c,d}, even-numbered columns are compared first, and odd-numbered columns are compared second. For example, comparing storage bubbles a and c, we find that the value of a (#3) is greater than the value of c (#1), so we swap them. The light-shaded line in the diagram represents the switch being on, allowing data to flow horizontally. Comparing storage bubbles b and d, we find that the value of b (#+∞) is greater than the value of d (#2), so we also swap them, the switch is on, and the data path flows horizontally. At this point, the sorting positions are c, d, a, b. Next, we compare adjacent storage bubbles. For example, comparing storage bubbles c and d, we find that the value of c (#1) is less than the value of d (#2), so we keep them unchanged, the switch is not on, and the data path can only flow vertically. Similarly, after comparing storage bubbles d and a, the switch is not on; after comparing storage bubbles a and b, the switch is not on.
[0172] Ultimately, it can be seen that each IP core corresponds exactly to the storage bubble it needs to access. For example, for IP#1, the path below it goes vertically downwards to the gray dot, then moves horizontally, and then vertically downwards to storage bubble c. The data paths for other IP cores are similar. Thus, based on the sorting network, a non-blocking data path is constructed between IP cores and storage bubbles.
[0173] Using a bitonic sorting network, we can achieve O(n(logk)) time complexity. 2 The Golgi is implemented using 1000 comparators and switches, which is much smaller than the O(nk) switches required for the cross matrix. Data delivered via the Golgi is subject to a delay of several cycles (e.g., 8 cycles), so it is preferable to place as little local cache as possible in the IP core (1KB is sufficient) since it relies on a large number of random accesses.
[0174] In summary, to execute a single MS instruction, SaaP establishes an exclusive data path between the IP core and its memory bubble. This exclusive data path in SaaP follows a PXO architecture and provides non-blocking data access with minimal hardware cost.
[0175] Data can be shared between IP cores by passing storage bubbles between MS instructions. Since the parent core manages system memory, input data is aggregated by the parent core within a single MS instruction and correctly placed into a storage bubble for use by another MS instruction. After being processed by the IP core, output data is similarly distributed back to system memory by the parent core. Specifically, the complete data path from system memory to the IP core includes: [(Loading MS instructions) ① Memory ② L3 / L2 cache ③ Parent core ④ Gorky W0 ⑤ Storage bubble, (Consuming MS instructions) ⑤ Same storage bubble ⑥ Gorky R0 / 1 ⑦ IP core.]
[0176] From a logical perspective, system memory is a resource exclusively owned by the parent kernel, which greatly reduces system complexity in the following ways:
[0177] 1) Page faults can only be initiated by the mother kernel and are handled within the mother kernel, so other MS instructions can always be executed safely under the condition that there are no page faults;
[0178] 2) The L2 / L3 cache is exclusively owned by the parent kernel, so cache inconsistency / contention / false sharing will never occur;
[0179] 3) Interrupts are always handled by the parent core, so other IP cores (literally) will not be interrupted.
[0180] programming
[0181] SaaP can adapt to various general-purpose programming languages (C, C++, Python, etc.) as well as domain-specific languages. Since any task executed on SaaP is an MS instruction, the key technology is to extract operations of mixed scale to form MS instructions.
[0182] Figure 12 An exemplary flowchart of a compilation method according to an embodiment of this disclosure is shown.
[0183] As shown in the figure, in step 1210, mixed-scale (MS) operations are extracted from the program to be compiled. These MS operations can have a variable number of execution cycles. Then, in step 1220, the extracted mixed-scale operations are encapsulated to form MS instructions.
[0184] Low-level operations can be extracted from basic instruction blocks, while high-level operations can be extracted in various ways, including but not limited to: 1) directly calling mappings from libraries, 2) refactoring from low-level program structures, and 3) manually setting compiler guidelines. Therefore, existing programs, such as deep learning applications written in Python using PyTorch, can be compiled to the SaaP architecture in a manner similar to multi-scalar channeling.
[0185] In some embodiments, the following five LLVM compilation passes can be optionally added to extend the traditional compiler.
[0186] a) Call-Map: This is a compilation process driven by a simple working list that translates known library calls into MS instructions. Specific implementations of these MS instructions are pre-compiled from vendor-specific code and referenced as libraries during the process.
[0187] Specifically, in one implementation, calls to library functions can be extracted from the program to be compiled as MS operations; then, based on the mapping list from library functions to the MS template library, the extracted calls to library functions are converted into corresponding MS instructions. The MS template library is pre-compiled based on the specific code of the execution unit capable of executing the library functions.
[0188] b) Reconstruct: This is another compilation process driven by a work list that attempts to recover high-level structure from low-level code, thus uncovering high-level MS instructions.
[0189] Specifically, in one implementation, a specified program structure in the program to be compiled is identified as an MS operation through template matching; and the identified specified program structure is converted into predetermined MS instructions. The template can be predefined based on the characteristics of high-level functional structures. For example, the template can define a nested loop structure and set some parameters of the nested loop structure, such as the number of nested loops, the size of each loop, and the operations in the innermost loop, etc. The template can be defined based on some typical high-level structures (e.g., convolution operation structures, Fast Fourier Transform (FFT), etc.), and the specific definition content and definition method are not limited in this disclosed embodiment.
[0190] For example, a user-implemented Fast Fourier Transform (FFT) (as a nested ring) can be captured via template matching and then replaced using the FFT MS instruction from a vendor-specific library used in the Call-Map. The recovered FFT MS instruction can be executed more efficiently on a DSP IP core (if available) and, in the worst case where only the CPU is available, can be converted back to a nested ring. This is done to the best of our ability, as accurately reconstructing all the high-level structures is inherently difficult, but it provides an opportunity for older programs unaware of DSA to utilize newer DSP IP cores.
[0191] c) CDFG (Control Data Flow Graph) Analysis Process: Unlike multiscalar techniques, the program is analyzed on a CDFG graph, not a CFG (Control Flow Graph). This is because SaaP removes register masking and address resolution mechanisms and organizes data into memory bubbles. After the previous two compilation processes, the operations to be executed on heterogeneous IP cores can be identified. All remaining code is to be executed on the CPU as multiscalar tasks. At this point, the problem is to find the optimal partitioning of the remaining code into MS instructions. A global CDFG is constructed for subsequent cost modeling of different MS instruction partitions.
[0192] Specifically, in one implementation, the operations that have not yet been extracted from the program to be compiled can be divided into one or more operation sets according to multiple partitioning schemes on the control data flow graph of the program to be compiled; then, the partitioning scheme with the optimal partitioning cost is determined. In each partitioning scheme, each operation belongs to one and only one operation set.
[0193] There are many ways to partition a data structure. Basically, a partitioning scheme can be executed by following one or more of the following constraints.
[0194] For example, the number of elements in the input and output data of an operation set cannot exceed a specified value. As specified by the MS instruction, the number of elements in the input data cannot exceed 2, and the number of elements in the output data cannot exceed 1. Therefore, operation partitioning can be performed based on this constraint.
[0195] For example, the size of any input or output data in an operation set must not exceed a specified threshold. Since the storage element corresponding to the MS instruction is a storage bubble, and the storage bubble has a capacity limit, it is necessary to limit the amount of data processed by the MS instruction to not exceed the capacity limit of the storage bubble.
[0196] For example, during partitioning, the partitioning schemes related to conditional operations can include:
[0197] 1. Conditional operations and their two branch operations should be grouped into a single operation set. In this case, the MS instruction corresponding to this operation set is a regular calculation instruction.
[0198] 2. The conditional operation and its two branch operations are not in the same operation set. Possible reasons for this splitting scheme include: it would result in an excessively large operation set; it would violate input / output constraints; or the branch operation had already been identified as an MS instruction in a previous step. In this case, an MS instruction of the branch type containing the conditional operation will be generated. Generally, placing the conditional operation in a short operation set allows for faster branching results during execution. For example, it's possible to control whether the same operation set contains both a conditional operation and an unconditional operation exceeding an execution time threshold simultaneously.
[0199] The partitioning cost of a partitioning scheme can be determined based on a variety of factors, including but not limited to: the number of operation sets; the amount of data interaction required between operation sets; the number of operation sets that perform branching functions; and the uniformity of the expected execution time distribution among the operation sets. These factors affect the execution efficiency of the instruction pipeline in multiple ways and can therefore serve as metrics for determining the partitioning scheme. For example, the number of operation sets directly corresponds to the number of MS instructions; the amount of data interaction required between operation sets determines the required data I / O; the more branching instructions there are, the greater the probability of triggering exceptions and the greater the pipeline overhead; and the uniformity of the expected execution time distribution affects the overall operation of the pipeline, preventing pipeline interruptions due to excessive time consumption at a certain stage.
[0200] In some embodiments, the CDFG analysis process described above is executed after the mapping and refactoring processes are invoked. Therefore, it can be executed only for MS operations not identified in the first two compilation processes, i.e., for the remaining operations.
[0201] d) MS-Cluster (MS set transformation process): This is a transformation compilation process used to aggregate nodes in CDFG to build a complete partition to MS instructions.
[0202] Specifically, in one implementation, each set of operations is converted into a single MS instruction based on the partitioning scheme determined during CDFG analysis. Limited by the capacity of the memory bubble, the algorithm minimizes the total cost of the cutting edges across MC instruction boundaries. In particular, MS instructions and system calls, including load / store operations, are assigned to the mother kernel.
[0203] e) Fractal-Decompose: This is also a transformation compilation process used to decompose MS instructions that violate the storage bubble capacity limit extracted from the call mapping process and refactoring process, so that the storage bubble capacity no longer limits the functionality of SaaP.
[0204] Specifically, in one implementation, the decomposition process includes: checking whether the converted MS instruction meets the storage capacity constraint of the MS instruction; when the MS instruction does not meet the storage capacity constraint, splitting the MS instruction into multiple MS instructions to achieve the same function.
[0205] Various existing or future instruction decomposition methods can be used to decompose MS instructions. Since the extracted MS instructions are intended to be executed on a single IP core, the multiple operations constituting these MS instructions are of the same type, i.e., isomorphic, only needing to adapt to the physical hardware size. Therefore, in some embodiments, the decomposition process of this MS instruction can simply follow a fractal execution model. For example, refer to the paper "Cambricon-F: Machine Learning Computers with Fractal von Neumann Architecture" by Y. Zhao, Z. Du, Q. Guo, S. Liu, L. Li, Z. Xu, T. Chen, and Y. Chen et al., in Proceedings of the 46th International Symposium on Computer Architecture, 2019, pp. 787–800. Generally, MS instructions can be decomposed into several smaller, similar operations through an iterative approach. Since the inventive point of this disclosure does not lie in a specific instruction decomposition method, it will not be described in detail here.
[0206] Encapsulating MS operations into MS instructions simply involves filling one or more instruction fields into the MS instruction. As mentioned earlier, MS instructions include a subinstruction field, an input / output memory bubble information field, and possibly a system memory address information field, a branch information field, and an exception flag field. Some of these instruction fields are mandatory, such as the subinstruction field and the exception flag field; others are filled as needed, such as the input / output memory bubble information field, the system memory address information field, and the branch information field.
[0207] When filling a subinstruction field, the MS operation can be identified in the subinstruction field of the MS instruction; and the subinstruction field can be associated with one or more execution unit-specific subinstructions used to implement the MS operation.
[0208] In some embodiments, for conditional computation MS instructions associated with branch MS instructions, a potentially discarded tag may be inserted in the exception tag field for use when executing MS instructions subsequently.
[0209] In some other embodiments, for MS instructions of branch type, a branch indicator can be inserted in the branch information field to indicate possible branch targets and / or impossible branch targets.
[0210] Figure 13 An example program is shown, in which (a) shows the original program to be compiled; the compiled program is divided into two parts, in which (b) shows the compiled MS instruction stream, and (c) shows the IP-specific MS instruction implementation, namely the sub-instructions described above.
[0211] In this example, the original program involves the computation of the ReLU and softmax layers of a neural network in a deep learning application, written in Python using PyTorch, for example. The computation of both the ReLU and softmax layers uses calls to the torch library. Therefore, following the call mapping process described earlier, these function calls to the torch library can be mapped to MS instructions, such as "matmul (matrix multiplication)," "eltwadd (element-wise addition)," "relu," etc., as shown in (b). The increment of the variable epoch and the conditional branch are packaged and mapped into a conditional branch instruction "ifcond," with branch indicators inserted to indicate possible and impossible branch targets. The print declaration is mapped to another MS instruction ("print").
[0212] (c) illustrates several MS instructions with IP-specific code. As shown, matmul provides two IP-specific code implementations, one for the GPU and the other for the DLA, allowing the "matmul" MS instruction to be scheduled between the GPU and DLA lanes by the instruction dispatcher. ifcond provides only CPU-specific code, which involves reading the epoch value from the first input storage bubble (vil), incrementing it by 1, and then storing it in the output storage bubble (vo). The result 10, modulo 10, of this new epoch value is then calculated and used for judgment. If it is determined that the "then" branch (which is compared as an impossible branch) should be taken, a "UBE" event is initiated. Therefore, the ifcond MS instruction also has a "possible discard tag" inserted, after which any subsequent large MS instructions will be blocked until the ifcond instruction has been executed. The print MS instruction is only dispatched to the mother core because this instruction requires system calls and I / O with external devices.
[0213] Therefore, the above describes an exemplary scheme for compiling program code into MS instructions. The program code to be compiled can be various general-purpose programming languages or domain-specific languages. By compiling this program code into MS instructions, various new IP cores can be easily added to the SaaP SoC without a large amount of programming / compilation work, thus providing excellent support for SoC scalability. In addition, the same MS instruction can use multiple versions of subinstructions, which also provides more options for instruction execution scheduling and facilitates improved pipeline execution efficiency.
[0214] In summary, SaaP offers a superior design option to the traditional understanding of heterogeneous SoCs. In SaaP, there is no contention due to the absence of shared resources under the PXO principle. MS instructions can be executed predictably and rolled back in case of errors without any overhead, as there are no observable side effects in the executing IP cores due to erroneous instructions. Caches do not need to maintain consistency because there are no duplicate cache lines, and snoop filter / MESI protocols are eliminated because there is no bus to snoop on. Although additional constraints are imposed on SaaP, it is clear from both analytical and empirical perspectives that these constraints are reasonable, as described in this paper.
[0215] Figure 14 A schematic diagram of a board 1400 according to an embodiment of this disclosure is shown. As shown, the board 1400 includes a chip 1401, which may be a SaaP SoC according to this disclosure embodiment. It integrates one or more combined processing devices, which are artificial intelligence computing units used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in the field of cloud intelligence. A significant characteristic of cloud intelligence applications is the large amount of input data, which places high demands on the platform's storage and computing capabilities. The board 1400 of this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and powerful computing capabilities.
[0216] Chip 1401 is connected to external device 1403 via external interface device 1402. External device 1403 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 1403 to chip 1401 via external interface device 1402. The calculation results from chip 1401 can be transmitted back to external device 1403 via external interface device 1402. Depending on the application scenario, external interface device 1402 may have different interface forms, such as a PCIe interface.
[0217] The board 1400 also includes a storage device 1404 for storing data, which includes one or more memory cells 1405. The storage device 1404 is connected to and transmits data with the controller 1406 and the chip 1401 via a bus. The controller 1406 in the board 1400 is configured to regulate the state of the chip 1401. Therefore, in one application scenario, the controller 1406 may include a microcontroller (MCU).
[0218] The SoC chip in the board provided in this disclosure embodiment may include the corresponding features described above, which will not be repeated here. This disclosure embodiment also provides a corresponding compilation apparatus, which includes a processor configured to execute compiler code; and a memory configured to store the compiler code. When the compiler code is loaded and executed by the processor, the compilation apparatus causes the compilation apparatus to execute the compilation method described in any of the preceding embodiments. This disclosure embodiment also provides a machine-readable storage medium including compiler code, which, when executed, causes the machine to execute the compilation method described in any of the preceding embodiments.
[0219] Depending on the application scenario, the electronic devices or apparatus disclosed herein may include servers, cloud servers, server computing clusters, data processing devices, robots, computers, printers, scanners, tablets, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The electronic devices or apparatus disclosed herein can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare. Furthermore, the electronic devices or apparatus disclosed herein can also be used in cloud, edge, and terminal applications related to artificial intelligence, big data, and / or cloud computing. In one or more embodiments, the high-computing-power electronic devices or apparatuses according to the present disclosure can be applied to cloud devices (e.g., cloud servers), while the low-power electronic devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling and collaborative work of end-to-cloud or cloud-edge-end integration.
[0220] It should be noted that, for the sake of brevity, this disclosure describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solutions disclosed herein are not limited by the order of the described actions. Therefore, based on the disclosure or teachings of this document, those skilled in the art will understand that some steps can be performed in a different order or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this disclosure can be considered optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions disclosed herein. In addition, depending on the solution, the description of some embodiments in this disclosure may have different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this disclosure can also be referred to the relevant descriptions of other embodiments.
[0221] In terms of specific implementation, based on the disclosure and teachings of this document, those skilled in the art will understand that several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various units in the electronic device or apparatus embodiments described above, this document has decomposed them based on logical functions, but in actual implementation, there may be other decomposition methods. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationships between different units or components, the connections discussed above in conjunction with the accompanying drawings can be direct or indirect couplings between units or components. In some scenarios, the aforementioned direct or indirect couplings involve communication connections utilizing interfaces, where the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.
[0222] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this disclosure. Additionally, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically independently.
[0223] In other implementation scenarios, the integrated units described above can also be implemented in hardware, i.e., as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the circuit's hardware structure may include, but is not limited to, physical devices, which may include, but are not limited to, transistors or memristors. Therefore, the various devices described herein (e.g., computing devices or other processing devices) can be implemented using appropriate hardware processors, such as central processing units, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage units or storage devices can be any suitable storage medium (including magnetic storage media or magneto-optical storage media), such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM.
[0224] The embodiments of this disclosure have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this disclosure. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this disclosure. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this disclosure. Therefore, the content of this specification should not be construed as a limitation of this disclosure.
Claims
1. An instruction processing apparatus, comprising: An instruction decoder is used to decode mixed-scale MS instructions, wherein the MS instructions include sub-instruction fields that indicate sub-instruction information specific to one or more execution units capable of executing the MS instructions. The execution units include multiple heterogeneous IP cores integrated on a system-on-a-chip (SoC). The multiple heterogeneous IP cores require different instruction sets. The MS instructions are unified instructions that can be applied to multiple heterogeneous IP cores and are independent of the IP cores. as well as An instruction dispatcher is used to dispatch the MS instruction to the corresponding execution unit according to the sub-instruction domain. Multiple heterogeneous IP cores are integrated into the SoC as execution units in the hardware pipeline to execute the dispatched instructions.
2. The apparatus of claim 1, wherein the plurality of execution units are divided into different lanes according to their functions, and The instruction dispatcher is further configured to: dispatch the MS instruction to a reserve station corresponding to a suitable lane, at least in part based on the task type of the MS instruction, for subsequent transmission to a suitable execution unit.
3. The apparatus of claim 2, wherein the instruction dispatcher is further configured to: schedule among lanes to which multiple execution units capable of executing the MS instructions belong, based on the processing status in the lane.
4. The apparatus according to any one of claims 1-3, wherein the instruction dispatcher is further configured to: The specified type of MS instruction is dispatched to the main execution unit responsible for management within the execution unit.
5. The apparatus of claim 4, wherein the specified type of MS instruction includes any of the following: MS instructions for accessing system memory; MS instructions for handling interrupts; MS instructions that cannot be processed by other execution units; MS instructions are allocated to the main execution unit according to the MS instruction scheduling policy.
6. The apparatus of claim 4, wherein one or more execution unit-specific sub-instructions are pre-fetched and stored in a sub-instruction buffer so that when the MS instruction is issued to the corresponding execution unit, the execution unit retrieves the corresponding sub-instruction from the sub-instruction buffer.
7. The apparatus according to any one of claims 1-3, wherein: The MS instruction of the arithmetic class performs operations on data stored in a set of storage bubbles, which are multiple temporary registers of different storage capacities.
8. The apparatus according to claim 7, further comprising: A storage bubble renaming circuit is used to rename and map the logical name of the storage bubble to a physical name before dispatching the MS instruction when a data hazard exists on the storage bubble involved in the MS instruction, and to save the mapping relationship between the physical name and the logical name.
9. The apparatus according to claim 8, further comprising: The instruction exit circuit is used to exit completed MS instructions in sequence, and submit the execution result by confirming the renaming mapping of the storage bubble corresponding to the output data of the MS instruction when the MS instruction exits.
10. The apparatus according to any one of claims 1-3, wherein each MS instruction has at most two input data fields and one output data field.
11. A method for executing instructions, comprising: The mixed-scale MS instruction is decoded. The MS instruction includes a sub-instruction field, which indicates sub-instruction information specific to one or more execution units that can execute the MS instruction. The execution unit includes multiple heterogeneous IP cores integrated on a system-on-a-chip (SoC). The multiple heterogeneous IP cores require different instruction sets. The MS instruction is a unified instruction that can be applied to multiple heterogeneous IP cores and is independent of the IP core. as well as According to the sub-instruction domain, the MS instruction is dispatched to the corresponding execution unit. Multiple heterogeneous IP cores are integrated into the SoC as execution units in the hardware pipeline to execute the dispatched instructions.
12. The method of claim 11, wherein the plurality of execution units are divided into different lanes according to their functions, and the assignment of the MS instructions to the corresponding execution units further comprises: Based at least in part on the task type of the MS command, the MS command is assigned to the appropriate reserve station corresponding to the appropriate lane for subsequent transmission to the appropriate execution unit.
13. The method of claim 12, wherein dispatching the MS instruction to the corresponding execution unit further comprises: Based on the processing status in the lane, scheduling is performed among the lanes to which multiple execution units capable of executing the MS instructions belong.
14. The method according to any one of claims 11-13, wherein dispatching the MS instruction to the corresponding execution unit further comprises: The specified type of MS instruction is dispatched to the main execution unit responsible for management within the execution unit.
15. The method of claim 14, wherein the specified type of MS instruction includes any of the following: MS instructions for accessing system memory; MS instructions for handling interrupts; MS instructions that cannot be processed by other execution units; MS instructions are allocated to the main execution unit according to the MS instruction scheduling policy.
16. The method of claim 14, further comprising: The one or more execution unit-specific sub-instructions are pre-fetched and stored in the sub-instruction cache; as well as When the MS instruction is issued to the corresponding execution unit, the execution unit retrieves the corresponding sub-instruction from the sub-instruction buffer.
17. The method according to any one of claims 11-13, wherein: The MS instruction of the arithmetic class performs operations on data stored in a set of storage bubbles, which are multiple temporary registers of different storage capacities.
18. The method of claim 17, further comprising: During the conflict resolution phase before dispatching the MS instruction, if a data hazard exists on a storage bubble involved in the MS instruction, the logical name of the storage bubble is renamed and mapped to its physical name. as well as Save the mapping relationship between the physical name and the logical name.
19. The method of claim 18, further comprising: When the MS instruction exits, the execution result is submitted by confirming the renaming mapping of the storage bubble corresponding to the output data of the MS instruction.
20. The method of claim 19, wherein the decoding, conflict resolution, dispatch, execution, and exit of the MS instruction are performed in parallel according to an out-of-order pipeline.
21. The method according to any one of claims 11-13, wherein each MS instruction has at most two input data fields and one output data field.
22. A system-on-a-chip (SoC) comprising an instruction processing apparatus according to any one of claims 1-10, and a plurality of heterogeneous IP cores, wherein the plurality of heterogeneous IP cores serve as the execution unit.
23. A board comprising the system-on-a-chip according to claim 22.
Citation Information
Patent Citations
Information processing method and terminal device
WO2019085655A1
