Integrated circuit and method for accelerating instructions
By introducing the accelerator data path into the CPU integrated circuit, the processor executes accelerating workload and stores the results, solving the problem of insufficient computing performance and power efficiency of traditional CPU architectures, realizing efficient neural network computing and enhancing the programmability of the system .
Patent Information
- Application Number
- CN202410586715.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-07-21
- Filing Date
- 2024-05-13
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional microprocessor/CPU architectures cannot provide sufficient computing performance and power efficiency to meet the high computing needs in machine learning, while graphics processors (GPUs) and custom neural processors (NPUs) are limited by programmability.
Design an accelerator system that is closely integrated into the CPU core, by introducing the accelerator data path into the CPU integrated circuit, using the accelerator processor to execute the acceleration workload encoded in the instructions, and store the results in embedded memory to improve the calculation Performance and power efficiency.
It realizes the calculation performance and power efficiency of neural network training and inference without affecting the normal operation of the CPU core, enhances the programmability of the system, and reduces the data transmission and intermediate results between the data path and the CPU memory. The dangers of writing and reading.
Smart Images

Figure CN119088536A_ABST
Abstract
Description
[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 471,443, filed on June 6, 2023, entitled “CPU Tightly Integrated Neural Network Accelerator,” and U.S. Application No. 18 / 225,041, filed on July 21, 2023, entitled “CPU Tightly Integrated Accelerator,” the entire contents of which are incorporated herein by reference. Technical Field
[0002] Aspects of embodiments of the present disclosure relate to accelerators, and more particularly, to accelerators tightly coupled to a central processing unit (CPU) core. Background Art
[0003] Machine learning typically involves training and inference as two main phases. During training, developers use curated data sets to train a neural network model so that the neural network model can learn as much as possible about the data it will analyze in order to make appropriate predictions. Once fully trained, the neural network model is able to make predictions based on actual real-time data during inference. Because neural network models typically need to calculate large amounts of data during training and inference, they may require processors with high computing performance, power efficiency, and programmability.
[0004] The above information disclosed in this Background section is for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute prior art. Summary of the invention
[0005] Embodiments of the present disclosure relate to systems and methods including an accelerator tightly coupled to a CPU core.
[0006] According to one or more embodiments of the present disclosure, an integrated circuit includes: a central processing unit (CPU) core; an accelerator; and an acceleration instruction queue connected to the CPU core and the accelerator. The CPU core is configured to: extract and decode one or more instructions from an instruction sequence in a programmed order; determine the instruction containing an acceleration workload encoded in the instruction from the one or more instructions; and queue the instruction containing the acceleration workload encoded in the instruction in the acceleration instruction queue.
[0007] In one embodiment, the accelerator may be configured to: dequeue the instruction containing the acceleration workload from the acceleration instruction queue; receive operands associated with the acceleration workload from a temporary memory of the CPU core; and calculate a result based on the operands and the dequeued instruction.
[0008] In one embodiment, the accelerator may be configured to dequeue instructions from the acceleration instruction queue in a first-in, first-out method.
[0009] In one embodiment, the accelerator may be further configured to store the result in an embedded memory of the accelerator.
[0010] In one embodiment, the CPU core, accelerator, temporary memory and embedded memory may be integrated with each other on the same chip.
[0011] In one embodiment, the CPU core may be configured to retrieve the results from the embedded memory of the accelerator and store the results in a temporary memory of the CPU core.
[0012] In one embodiment, the accelerator instruction queue may include a plurality of instruction queues having different priorities defined for the accelerator.
[0013] According to one or more embodiments of the present disclosure, a computing system includes: an accelerator; one or more processors integrated with the accelerator in the same integrated circuit; and a memory, including instructions stored on the memory, which, when executed by the one or more processors, enable the one or more processors to: identify a programming order for executing one or more CPU instructions; and execute the one or more CPU instructions according to the programming order. To execute the one or more CPU instructions, the instructions enable the one or more processors to: extract and decode a first instruction from the one or more CPU instructions in the programming order; and, based on the instruction type of the first instruction, dispatch the decoded first instruction to a CPU data path or an accelerator data path from a CPU pipeline.
[0014] In one embodiment, the first instruction may include an accelerator workload encoded in the first instruction to be dispatched to the accelerator data path, and the instruction may also cause one or more processors to: enqueue the first instruction in an acceleration instruction queue; and provide corresponding operands to the accelerator for calculation based on the first instruction.
[0015] In one embodiment, the accelerator may be configured to: dequeue a first instruction from an acceleration instruction queue; calculate a result based on the corresponding operand and the first instruction dequeued from the acceleration instruction queue; and store the result in an embedded memory of the accelerator.
[0016] In one embodiment, the accelerator may be configured to dequeue instructions from the acceleration instruction queue in a first-in, first-out manner.
[0017] In one embodiment, the instructions may further cause the one or more processors to: retrieve the result from the embedded memory of the accelerator; and store the result in a temporary memory.
[0018] In one embodiment, the accelerator, the one or more processors, the embedded memory and the temporary storage may be integrated into the same integrated circuit.
[0019] In one embodiment, a first instruction may be dispatched to a CPU data path, and the instruction may also cause one or more processors to: extract and decode a second instruction from the one or more CPU instructions in the programmed order; determine an acceleration workload encoded in the second instruction; enqueue the second instruction in an acceleration instruction queue; and provide corresponding operands to an accelerator for calculation based on the second instruction.
[0020] According to one or more embodiments of the present disclosure, a method for accelerating instructions includes: identifying, by one or more processors, a programming order for executing one or more instructions; determining, by the one or more processors, an acceleration workload encoded in the instructions in the one or more instructions in the programming order; and dispatching, by the one or more processors, the instructions to an accelerator data path among multiple data paths of a CPU pipeline based on determining that the acceleration workload is encoded in the instructions.
[0021] In one embodiment, the dispatching step may include: enqueuing, by the one or more processors, the instruction in an acceleration instruction queue; and providing, by the one or more processors, corresponding operands to an accelerator data path for calculation based on the instruction.
[0022] In one embodiment, the accelerator data path may include an accelerator integrated in the same integrated circuit as the one or more processors, and the method may also include: the accelerator dequeuing the instruction from an accelerated instruction queue; the accelerator calculating a result based on the corresponding operands and the instruction dequeued from the accelerated instruction queue; and the accelerator storing the result in an embedded memory of the accelerator.
[0023] In one embodiment, the accelerator may be configured to dequeue instructions from the acceleration instruction queue in a first-in, first-out manner.
[0024] In one embodiment, the method may further include: retrieving, by the one or more processors, the result from an embedded memory of an accelerator; and storing, by the one or more processors, the result in a temporary memory of the one or more processors.
[0025] In one embodiment, the accelerator and the one or more processors may be a co-processor or multi-processors of the same integrated circuit. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The above and other aspects and features of the present disclosure will be more clearly understood from the following detailed description of illustrative, non-limiting embodiments with reference to the accompanying drawings, in which:
[0027] Figure 1 is a schematic block diagram of a computing system according to one or more embodiments of the present disclosure;
[0028] Figure 2A and Figure 2B is a schematic block diagram of a CPU integrated circuit according to one or more embodiments of the present disclosure;
[0029] Figure 3 is a schematic block diagram of a CPU pipeline according to one or more embodiments of the present disclosure;
[0030] Figure 4 is a schematic block diagram of an accelerator according to one or more embodiments of the present disclosure;
[0031] Figure 5 is a schematic block diagram of an acceleration operation according to one or more embodiments of the present disclosure;
[0032] Figure 6 is a schematic block diagram of a processing element according to one or more embodiments of the present disclosure;
[0033] Figure 7 shows example instructions for acceleration according to one or more embodiments of the present disclosure;
[0034] Figure 8 A flowchart showing a method for accelerating CPU instructions according to one or more embodiments of the present disclosure;
[0035] Fig. 9 A flowchart showing a method for selectively invoking an accelerator for an acceleration workload encoded in CPU instructions executed in a programmed order according to one or more embodiments of the present disclosure; and
[0036] Fig.10 A flow chart of a method for processing an acceleration workload from an acceleration instruction queue according to one or more embodiments of the present disclosure is shown. DETAILED DESCRIPTION
[0037] Hereinafter, embodiments will be described in more detail with reference to the accompanying drawings, wherein the same reference numerals represent the same elements throughout. However, the present disclosure may be implemented in various different forms and should not be construed as being limited to the embodiments shown herein. On the contrary, these embodiments are provided as examples so that the present disclosure will be comprehensive and complete, and the aspects and features of the present disclosure will be fully conveyed to those skilled in the art. Therefore, processing, elements and techniques that are unnecessary for a complete understanding of the aspects and features of the present disclosure for those of ordinary skill in the art may not be described. Unless otherwise stated, throughout the accompanying drawings and written descriptions, the same reference numerals represent the same elements, and therefore, their redundant descriptions may not be repeated.
[0038] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those of ordinary skill in the art to which the present disclosure belongs. It will also be understood that, unless expressly defined herein, terms (such as those defined in general dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and should not be interpreted in an idealized or overly formal sense.
[0039] Neural network training and inference may require high computing performance, power efficiency, and programmability. However, for typical artificial intelligence / machine learning (AI / ML) workloads, traditional microprocessor / CPU architectures may not provide sufficient computing performance or power efficiency. On the other hand, typical graphics processing unit (GPU) architectures and custom-designed neural processing unit (NPU) architectures may each be affected by programmability - often requiring complex software stacks and heterogeneous programming models.
[0040] One or more embodiments of the present disclosure may be directed to a CPU instruction-based neural network accelerator (e.g., such as for model training and / or inference) with high computing performance, programmability, and power efficiency. For example, in some embodiments, a neural network acceleration engine may be integrated with a general-purpose CPU core and may be diverted to accelerate a workload (or accelerate a workload) as needed or desired, which may allow the CPU core to perform other tasks concurrently or substantially simultaneously with it.
[0041] In some embodiments, the acceleration engine may be integrated into the CPU data path that will be used for the accelerated workload. For example, in some embodiments, the accelerated workload may be compiled into a CPU instruction sequence and assigned to the accelerated data path (or accelerator data path) as needed or desired. In this way, in addition to the typical data path (e.g., scalar data path, vector data path, memory data path, etc.) of the CPU pipeline for the CPU core, the accelerated data path may provide different data paths or separate data paths to the CPU pipeline to process the accelerated workload, and may be called (e.g., used) as needed or desired while having minimal impact on the operation and processing of the CPU core and the CPU core. For example, the accelerated data path may be used only by those CPU instructions in the CPU instruction sequence that include the accelerated workload, and may not be used by other CPU instructions in the CPU instruction sequence that include other typical CPU workloads (e.g., scalar workloads, vector workloads, memory workloads, etc.).
[0042] In some embodiments, the intermediate results (e.g., intermediate outputs or partial outputs) of the acceleration data path may be saved in a memory or storage device (e.g., such as a register file) embedded in the acceleration data path, and thus, the results of the acceleration data path may be stored separately from the memory or storage device of the CPU core, and may be retrieved by the CPU core as needed or desired (e.g., read by or sent to the CPU core). In some embodiments, the intermediate outputs or partial outputs of the acceleration data path may first be temporarily accumulated in an accumulation buffer during multiple periods or multiple stages of the acceleration data path, and in the final period or final stage of the acceleration data path, the results may be submitted to the embedded memory or storage device. Therefore, the input / output (I / O) transmitted between the processing element of the acceleration data path and the embedded memory or storage device (e.g., such as during the calculation of the acceleration workload during multiple periods or multiple stages of the acceleration data path) may be reduced, and the read-after-write (RAW) data hazards of the intermediate results during the calculation during multiple periods or multiple stages may be reduced.
[0043] In some embodiments, the acceleration engine may be integrated with multiple CPU cores so that each of the multiple CPU cores can use the acceleration engine as needed or desired. For example, in some embodiments, the acceleration engine may be configured for parallel processing so that the acceleration engine can concurrently or substantially simultaneously calculate or process acceleration workloads from two or more CPU cores. Thus, the utilization of the acceleration engine can be increased (e.g., such as in situations where a single CPU core may not be able to fully utilize the performance and / or bandwidth of the acceleration engine).
[0044] The above and other aspects and features of the present disclosure will now be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are described in the context of AI / ML neural networks, the present disclosure is not limited thereto, and the CPU / accelerator architecture according to one or more embodiments of the present disclosure may be applicable to any suitable system or network that may benefit from an accelerator tightly coupled to a CPU as described herein.
[0045] Figure 1 is a schematic block diagram of a computing system according to one or more embodiments of the present disclosure.
[0046] First, refer to Figure 1 , the computing system 100 may include a main CPU 102, a CPU integrated circuit 104, and a shared memory 106. The CPU integrated circuit 104 may be a processing circuit including, for example, a digital circuit (e.g., a microcontroller, a microprocessor, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.). The CPU integrated circuit 104 may be, for example, a system on a chip (SOC) including one or more CPU cores 108, a temporary memory (SCM) 110, and a tightly coupled (e.g., integrated) accelerator 114 including an accelerator memory 116. As used herein, the term "tightly coupled" may mean that the accelerator 114 is integrated on the same chip with the one or more CPU cores 108. In other words, in some embodiments, the CPU core 108, the accelerator 114, the SCM 110, and the accelerator memory 116 may be included on the same chip and / or in the same package as each other.
[0047] The main CPU 102, the CPU core 108, and the accelerator 114 may each be implemented using a general-purpose processor, an ASIC, one or more FPGAs, a DSP, a group of processing components, or other suitable electronic processing components capable of executing instructions (e.g., via firmware and / or software). The main CPU 102 may be responsible for main CPU functions and operations (e.g., such as running applications and operating system (OS) functions and operations). The CPU core 108 may support the main CPU 102 with specific operations and functions (e.g., such as arithmetic operations and calculations). The accelerator 114 may support the CPU core 108 with accelerated operations (e.g., such as tensor operations (e.g., tensor multiplication)). In other words, in some embodiments, the CPU core 108 and the accelerator 114 may be understood as a coprocessor or multiprocessor of the same CPU integrated circuit 104. For example, in some embodiments, the accelerator 114 may essentially be understood as a collection of multiple multiplication units and accumulation (MAC) units and associated storage registers (e.g., accelerator memory 116), so that the accelerator 114 may form a large register file (e.g., see Figure 5The accelerator 114 and its operation will be described in more detail below.
[0048] Although Figure 1 The main CPU 102 is shown as being external to the CPU integrated circuit 104, but the present disclosure is not limited thereto. In other embodiments, the main CPU 102 may be a CPU core included in (e.g., integrated into) an integrated circuit 104 (e.g., such as in a dual-core processor or a multi-core processor) with an accelerator 114 and other CPU cores (e.g., one or more CPU cores 108). In some embodiments, the CPU integrated circuit 104 may include, for example, Figure 2A The single CPU core 108 shown in FIG. 1 may also include a CPU core 108 as shown in FIG. Figure 2B The multiple CPU cores 108_1 to 108_n (where n is a natural number greater than 1) are shown in FIG.
[0049] In some embodiments, the main CPU 102 and the CPU core 108 may be connected to the shared memory 106. The shared memory 106 may be a pool of memory devices (e.g., memory chips), and may be internal memory relative to the main CPU 102, the CPU core 108, or internal memory relative to both the main CPU 102 and the CPU core 108. In some embodiments, the shared memory 106 may include a plurality of distributed memories (e.g., memory devices or chips), each of which is connected to a corresponding CPU core and logically shared between the main CPU 102 and the CPU core 108. The SCM 110 may be an internal memory relative to the CPU core 108, and the accelerator memory 116 may be an internal memory relative to the accelerator 114. However, the shared memory 106 may have a larger capacity than the capacity of the SCM 110 and / or the accelerator memory 116 (e.g., 16 kB (kilobytes) or 32 kB).
[0050] For example, in various embodiments, the shared memory 106, the SCM 110, and the accelerator memory 116 may each include one or more random access memory (RAM) elements (such as static RAM (SRAM)), but the present disclosure is not limited thereto. In various embodiments, the shared memory 106, the SCM 110, and the accelerator memory 116 may include any suitable memory device (e.g., such as SRAM, dynamic RAM (DRAM), relatively high-performance non-volatile memory (e.g., NAND flash memory, phase change memory (PCM), resistive RAM, spin-transfer torque RAM (STTRAM), any suitable memory based on PCM technology, memristor technology, and / or resistive random access memory (ReRAM))), and may include, for example, chalcogenides, etc.
[0051] In short, in some embodiments, the main CPU 102 may submit commands and corresponding data (e.g., operands) to the shared memory 106. For example, in some embodiments, the main CPU 102 may have multiple applications running on the CPU 102, and some of the applications may send commands to be processed by the CPU core 108. The CPU core 108 may retrieve the command from the shared memory 106, and may store the command and corresponding data in the SCM 110. While executing the CPU instruction sequence based on the command, the CPU core 108 may determine a CPU instruction (e.g., a CPU acceleration instruction) from among the CPU instruction sequences corresponding to the accelerated workload suitable for processing by the accelerator 114. In this case, the CPU core 108 may transmit (e.g., may move) the instruction (e.g., the CPU acceleration instruction) to the acceleration instruction queue (AIQ) 112, and may provide the corresponding acceleration operand (e.g., a data point or value stored in a register file) in the SCM 110 to the accelerator 114 for calculation. The result of the calculation may be stored in the accelerator memory 116, and may be retrieved (e.g., read by or sent to the CPU core 108) by the CPU core 108 as needed or desired to be provided to the shared memory 106 for access by the main CPU 102 (e.g., by an application running on the CPU 102).
[0052] In some embodiments, the accelerator 114 may be invisible to standard applications that do not require acceleration, but standard applications that do not require acceleration may still use standard operations (e.g., scalar operations and vector operations) of the CPU core 108. In some embodiments, the application may access the accelerator 114 through a standard library (e.g., such as BLAS, OpenCL, etc.) and / or through specific custom instructions. For example, in some embodiments, the accelerator 114 may be accessed by calling a subroutine (e.g., a specific subroutine) by the CPU core 108, but the present disclosure is not limited thereto.
[0053] Figure 2A and Figure 2B is a schematic block diagram of a CPU integrated circuit according to one or more embodiments of the present disclosure.
[0054] like Figure 2A As shown in FIG. 1 , the CPU integrated circuit 104 may include a single CPU core 108 connected to an accelerator 114 via an AIQ 112, or as shown in FIG. Figure 2BAs shown in , the CPU integrated circuit 104 may include a plurality of CPU cores 108_1 to 108_n, each of which is connected to an accelerator 114 via a corresponding one of a plurality of AIQs 112_1 to 112_n (where n is a natural number greater than 1). For example, in the case where a single CPU core may not be able to fully utilize the performance and / or bandwidth of the accelerator 114, Figure 2B The embodiments shown in may be desirable such that the accelerator 114 may concurrently or substantially simultaneously compute or process acceleration workloads from multiple CPU cores 108_1 to 108_n.
[0055] Each of the plurality of CPU cores 108_1 to 108_n may be the same as or substantially the same as the CPU core 108, and each of the plurality of AIQs 112_1 to 112_n may be the same as or substantially the same as the AIQ 112. Therefore, the CPU core 108 and the AIQ 112 may be described in more detail below, and redundant descriptions regarding the plurality of CPU cores 108_1 to 108_n and the plurality of AIQs 112_1 to 112_n may not be repeated.
[0056] like Figure 2A and Figure 2B As shown in , the CPU accelerates instructions and sends instructions from SCM 110 (see, for example, Figure 1 ) may be queued by the CPU core 108 in the AIQ 112. In some embodiments, a single CPU acceleration instruction may be divided into multiple micro-operations (μOp) and queued in the AIQ 112. The accelerator 114 may dequeue the CPU acceleration instruction from the AIQ 112, and execute the CPU acceleration instruction in the order in which the CPU acceleration instruction is dequeued from the AIQ 112. For example, the instruction may be dequeued from the AIQ 112 in a first-in-first-out (FIFO) method, but the present disclosure is not limited thereto. In some embodiments, the AIQ 112 may be implemented as a shallow buffer (e.g., such as a four-slot flip-flop, etc.), but the present disclosure is not limited thereto. The acceleration result is stored in, for example, an accelerator memory 116 (e.g., see ) embedded in the accelerator 114 as a register file (e.g., a tensor register file) TRF. Figure 1 Then, the CPU core 108 can read the contents of the register file TRF back to its SRF through specific instructions, which will be described in more detail below.
[0057] In some embodiments, AIQ 112 (e.g., each of AIQ 112_1 to 112_n) may include multiple queues, which may allow concurrent or substantially simultaneous execution when multiple data processing units (e.g., accelerator units) are present in accelerator 114. In some embodiments, multiple queues of AIQ 112 (e.g., each of AIQ 112_1 to 112_n) may provide a priority order of queues, so that accelerator 114 may preferentially execute CPU acceleration instructions queued in a higher priority queue in AIQ 112. For example, one of the queues in AIQ 112 may be a latency-sensitive queue for those time-critical acceleration operations, while another queue in AIQ 112 may be a throughput-oriented queue for those acceleration operations that only utilize the increased throughput provided by accelerator 114. In some embodiments, because the acceleration operations can be dequeued from each of the queues in the order in which they were enqueued (e.g., FIFO), the execution order of the acceleration operations from any one of the queues of AIQ 112 can be guaranteed, but the execution order of the acceleration operations may not be guaranteed when the acceleration operations are enqueued in different queues.
[0058] Figure 3 is a schematic block diagram of a CPU pipeline according to one or more embodiments.
[0059] Reference Figures 1 to 3 , the CPU pipeline 300 may include a typical CPU data path 302 (e.g., such as a typical CPU data path 302 for typical scalar operations, memory operations, vector operations, etc.). For example, Figure 3 Non-limiting examples of scalar operations and memory operations are shown as part of the CPU data path 302. However, unlike a typical CPU pipeline, the CPU pipeline 300 according to one or more embodiments of the present disclosure may also include a separate acceleration data path 304. For example, Figure 3 304, but the present disclosure is not limited thereto. Each of data paths 302 and 304 may include, for example, Figure 3 , so that one or more operations (e.g., fetch, decode, encode, read register file, calculate, write register file, etc.) are performed during each period or each stage. For example, each CPU instruction may be instruction fetched and instruction decoded by the CPU core 108, and may be dispatched to an appropriate one of the data paths based on the availability of the data path and the type of the CPU instruction (e.g., scalar, accelerated, etc.).
[0060] For example, in some embodiments, the acceleration workload may be encoded into a CPU instruction, fetched, and decoded in a typical manner in one or more periods or one or more stages (e.g., I$TLB, I$TAG, I$(R), DC, RF(R), etc.) of the CPU datapath 302. However, unlike other CPU instructions, the CPU instruction encoded with the acceleration workload (e.g., CPU acceleration instruction) is dispatched (e.g., via AIQ 112) to an integrated accelerator unit (e.g., see FIG. 114 ) of an accelerator 114 in the acceleration datapath 304. Figure 4 ). The CPU acceleration instruction may be executed in the accelerator unit in multiple time periods or multiple stages (e.g., SRF(R), TIQ, ALU, etc.) of the acceleration data path 304, and finally, the result or partial product of the accelerator unit is submitted to the TRF (e.g., accelerator memory 116). The CPU acceleration instruction may not cause any recoverable exceptions in the accelerator unit, but the CPU core 108 can generally handle typical exceptions (such as debug, virtual memory trap, etc.). In addition, even if the CPU acceleration instruction is executed during multiple time periods or multiple stages in the accelerator data path 304, there may be minimal impact on other operations of the CPU core 108 (e.g., standard scalar operations).
[0061] In other words, each CPU instruction from the instruction sequence executed by the CPU core 108 may be executed through multiple periods or multiple stages of the CPU pipeline 300. After the CPU instruction passes through multiple periods or multiple stages of the CPU data path 302 (e.g., extracted and decoded by the CPU core 108) (e.g., processed through multiple periods or multiple stages of the CPU data path 302 (e.g., extracted and decoded by the CPU core 108)), the CPU core 108 may identify the acceleration workload encoded in the CPU instruction, and may transmit (e.g., send or move) the CPU instruction with the acceleration workload (e.g., the CPU acceleration instruction) to the AIQ 112, and provide the corresponding acceleration operand to the accelerator unit in the acceleration data path 304 for processing (e.g., reading operands from the register file, dequeuing the CPU acceleration instruction from the AIQ, calculating, etc.) through multiple periods or multiple stages of the acceleration data path 304. The results or partial products of the accelerator units in the accelerated data path 304 (the results or partial products of the accelerator units in the accelerated data path 304 that may be calculated during multiple periods or multiple stages) are ultimately submitted to the final stage of the accelerated data path in the TRF and may be read into the SRF by the CPU core 108 as needed or desired. The TRF is Figure 3 3 is shown as the final stage of the accelerated data path 304 .
[0062] As an illustrative example, the instruction sequence may include a first memory load instruction, a second memory load instruction, and a tensor multiplication instruction. These instructions of the instruction sequence may be executed sequentially so that each of the instructions can be executed through multiple time periods or multiple stages (e.g., fetch, decode, etc.) of the CPU pipeline 300. In this example, since the first memory load instruction and the second memory load instruction do not include any accelerated workloads encoded therein, those instructions may be sent to other data paths of the CPU data path 302 (e.g., memory operations) to be executed through multiple time periods or multiple stages of other data paths of the CPU data path 302. On the other hand, during the time periods or stages (e.g., Figure 3 After the tensor multiplication instruction is fetched and decoded in the appropriate time slot or stage in the AIQ 112 (I$TLB, I$TAG, I$(R), DC, and RF(R) in the example shown in ), the accelerator 114 can be used to execute the tensor multiplication instruction through multiple time slots or multiple stages (e.g., SRF(R), TIQ, ALU, etc.) of the acceleration data path 304, and finally (e.g., in the last stage) the result of the tensor multiplication instruction is submitted to the TRF of the accelerator memory 116. For example, in some embodiments, after the CPU core 108 fetches and decodes the CPU acceleration instruction, the CPU core 108 may call a specific subroutine to use the accelerator 114, but the present disclosure is not limited thereto. Once the tensor multiplication instruction is queued in the AIQ 112, even during the multiple time slots or multiple stages that the accelerator 114 takes to complete the tensor multiplication instruction, the CPU core 108 is free to handle or process other instructions (e.g., scalar operations, etc.) for its typical CPU data path 302.
[0063] Figure 4 is a schematic block diagram of an accelerator according to one or more embodiments of the present disclosure. Figure 5 is a schematic block diagram of an acceleration operation according to one or more embodiments of the present disclosure. Figure 6 is a schematic block diagram of a processing element according to one or more embodiments of the present disclosure. Figure 7 Example instructions for acceleration according to one or more embodiments of the present disclosure are shown.
[0064] Reference Figures 4 to 6In some embodiments, the accelerator 114 may include one or more accelerator units 402, 404, ..., 406. Each of the accelerator units 402, 404, ..., 406 may include one or more sub-units (e.g., such as a specific function (SF) sub-unit (e.g., an exponential function (e.g., exp(x)), a sigmoid function (e.g., tanh), GELU, Softmax, etc.), a matrix multiplication (MM) sub-unit, etc.). Each of the sub-units may include a plurality of processing elements PE, and each of the processing elements PE may include a plurality of MAC units for performing multiplication and accumulation processing. For example, Figure 5 The MM subunit for 4×4 tensor multiplication operations shown in FIG. 1 may include 16 processing elements P 00 To P 15 An array for computing data along 2 dimensions (eg, rows and columns). As another example, an MM subunit for an 8×8 tensor multiplication operation may include an array of 64 processing elements PEs.
[0065] Figure 5 A non-limiting example of a 4×4 tensor multiplication is shown as an accelerated operation. Figure 5 As shown in FIG. 1 , the MM subunit of the accelerator 114 may be configured to calculate two vector registers V of the SCM 110. 1 and V 2 The matrix product of the processing element P 00 To P 15 Each of the vector registers V can be configured to use an outer product to perform 1 and V 2 In some embodiments, the processing element P 00 To P 15 The partial output or intermediate output of each of the can be accumulated to the accumulation buffer ACC in the acceleration data path 304 to be temporarily stored in the accumulation buffer ACC during the multiple periods or multiple stages taken to complete the CPU acceleration instruction, and finally, the final result of the calculation can be submitted to the appropriate part (e.g., register T1) of the accelerator memory 116 (e.g., TRF) during, for example, the last stage of the acceleration data path.
[0066] and Figure 5 Some example instructions associated with the tensor multiplication operation shown in Figure 7 Reference Figure 5 and Figure 7 , X0 to X3 and Y0 to Y3 are vector registers V from SCM 110 1 and V 2, and T1 is a tensor register of the accelerator memory 116 (e.g., TRF), such that the accelerator memory 116 may include multiple registers (e.g., multiple tensor registers). In other words, for illustrative purposes, Figure 5 and Figure 7 Shows the vector register V 1 With V 2 An example of matrix multiplication between processing elements P 00 To P 15 The results of each calculation during multiple periods or multiple stages of the acceleration data path are finally submitted to the TRF of the accelerator memory 116 to form the tensor register T1. 00 To P 15 Each of the V 1 The operands and the corresponding V 2 The product of the operands. For example, Figure 5 As shown in FIG. 1 , the processing element P 05 The product between X1 and Y1 may be calculated, which may require multiple periods or multiple stages to complete, and its result (eg, final result) may be submitted to the embedded tensor register T1.
[0067] In some embodiments, because the tensor multiplication may be calculated during multiple periods or multiple stages of the acceleration data path 304, the tensor multiplication may be calculated by the processing element P during multiple periods or multiple stages. 00 To P 15 In this case, if partial or intermediate I / Os are provided to and from tensor register T1 for each of a plurality of periods or a plurality of stages, accelerator memory 116 may be overburdened by I / O requests between stages, which may increase the latency of CPU acceleration operations. According to one or more embodiments of the present disclosure, instead of submitting partial outputs or intermediate outputs of each of processing elements PE to tensor register T1 during each period or stage of the acceleration data path 304, processing element P 00 To P 15 The intermediate output or partial output may be temporarily stored in the accumulation buffer ACC. Once the calculation may include the processing element P 00 To P 15 The final result of each output intermediate result or partial result in the accelerator data path 304 can be submitted to the tensor register T1 in the final stage (for example, in the final period or final stage of the accelerator data path 304).
[0068] In some embodiments, a processing element PE (e.g., Figure 5 The processing element P in 00 To P 15 Each of the processing elements PE (e.g., each of the processing elements PE) may include an accumulation buffer ACC for storing partial output or intermediate output. For example, in some embodiments, the accumulation buffer ACC may be implemented as a plurality of flip-flops or a plurality of registers, but the present disclosure is not limited thereto. In some embodiments, to avoid RAW data hazards, the processing element PE (e.g., each of the processing elements PE) may include an accumulation buffer ACC for storing partial output or intermediate output. Figure 5 The processing element P in 00 To P 15 Each of the Figure 6 In some embodiments, multiple accumulation buffers ACC for storing partial outputs or intermediate outputs are shown in FIG. 00 To P 15 All accumulation buffers ACC are collected to form tensor registers T1 of the tensor register file TRF. For example, for an 8×8 array of processing elements PE, the size of the total accumulation buffer ACC may be 4KB (eg, 8×8×16×4=1024×4=4KB).
[0069] In some embodiments, once the final output stored in the accumulation buffer ACC of each of the processing elements PE is submitted to the tensor register T1, the data in the tensor register T1 (e.g., the final result) may be moved back to the register (e.g., the vector register) of the SCM 110. In the illustrative example, because the tensor register T1 may be larger than the vector register size of the SCM 110, multiple vector registers of the SCM 110 may be used to hold the data from the tensor register T1. In other words, in some embodiments, a slice of a TRF in the accelerator memory 116 containing multiple tensor registers (e.g., the tensor register T1) may be moved from the accelerator memory 116 to N registers (e.g., N vector registers) of the SCM 110, where N is a natural number greater than 1. In some embodiments, once the data in the tensor register T1 is moved to the SCM 110, the data may be moved from the SCM 110 to the shared memory 106 by the CPU core 108 to be accessed by the host CPU 102 (e.g., by a requesting application running on the host CPU 102).
[0070] Figure 8 A flowchart of a method for accelerating CPU instructions according to one or more embodiments of the present disclosure is shown. However, the present disclosure is not limited to Figure 8The order or number of operations of the method 800 shown in the embodiment of the present invention is not limited to the order or number of operations, and may be changed to any desired order or number of operations as recognized by those of ordinary skill in the art. For example, in some embodiments, the order thereof may be changed, some of the processes thereof may be performed simultaneously or sequentially, or the method 800 may include fewer or additional operations.
[0071] Reference Figure 8 , method 800 may begin, and a command may be received at block 805. For example, in some embodiments, CPU core 108 may receive a command from host CPU 102 (e.g., from an application running on host CPU 102) via shared memory 106. In some embodiments, CPU core 108 may store the command and corresponding operands retrieved from shared memory 106 in SCM 110 (e.g., in corresponding registers of the SRF).
[0072] At block 810, one or more CPU instructions may be executed in a programmed order according to the command. For example, in some embodiments, to execute the command, the CPU core 108 may sequentially execute one or more CPU instructions in a programmed order. In this case, some of the CPU instructions in the programmed order may contain accelerated workloads encoded therein, while other of the CPU instructions in the programmed order may contain other typical workloads (e.g., scalar workloads, vector workloads, etc.).
[0073] At block 815, an acceleration workload encoded in an instruction in one or more CPU instructions in a programmed order may be identified. For example, when each of the CPU instructions is fetched and decoded in a programmed order, the CPU core 108 may identify an acceleration workload encoded in at least one of the CPU instructions based on the instruction type. In this case, at block 820, the instruction with the acceleration workload encoded in the instruction and the corresponding operands for calculation may be provided to the accelerator. For example, as described above, in some embodiments, the CPU core 108 may enqueue the instruction with the acceleration workload encoded therein in a suitable AIQ 112, and may provide the corresponding operands stored in the SCM 110 to the accelerator 114 for calculation based on the instruction with the acceleration workload enqueued in the AIQ 112.
[0074] In block 825, the result of the accelerator for accelerating the workload may be retrieved from the accelerator memory. For example, in some embodiments, the CPU core 108 may be able to determine that the accelerated instruction is executed by viewing the AIQ 112. For example, if the AIQ 112 is empty, the CPU core 108 may determine that all previous instructions enqueued in the AIQ 112 have been completed. In some embodiments, to alleviate scheduling requirements, the CPU core 108 may assume that the accelerated instruction is completed in the AIQ once it is enqueued. In another example, after the appropriate number of time periods or stages of the accelerated data path have passed, the CPU core 108 may read the result from the corresponding register (e.g., T1 of the TRF) of the accelerator memory 116 storing the result. As another example, in some embodiments, when the result is submitted to the accelerator memory 116, a notification may be provided to the CPU core 108. In another example, in some embodiments, once the result is submitted to the accelerator memory 116, the accelerator 114 may provide the result to the CPU core 108.
[0075] At block 830, the results may be stored in the CPU memory and the method 800 may end. For example, in some embodiments, the CPU core 108 may transfer the results from the SCM 110 to the shared memory 106 (e.g., may move the results from the SCM 110 to the shared memory 106) so that the host CPU 102 (e.g., a requesting application running on the host CPU 102) may access the results from the shared memory 106.
[0076] The following will refer to Fig. 9 An example method performed by the CPU core 108 according to the operations of blocks 810, 815, and 820 of the method 800 is described in more detail. Fig.10 An example method performed by the accelerator 114 in accordance with the operations of block 820 of the method 800 is described in further detail.
[0077] Fig. 9 A flowchart of a method for selectively invoking an accelerator for an acceleration workload encoded in CPU instructions executed in a programmed order according to one or more embodiments of the present disclosure is shown. However, the present disclosure is not limited to Fig. 9 The order or number of operations of the method 900 shown in the embodiment of the present invention is not limited to the order or number of operations, and may be changed to any desired order or number of operations as recognized by those of ordinary skill in the art. For example, in some embodiments, the order thereof may be changed, some of the processes thereof may be performed simultaneously or sequentially, or the method 900 may include fewer or additional operations.
[0078] Reference Fig. 9, method 900 may begin, and at block 905, a programming order for executing one or more CPU instructions may be identified. For example, in some embodiments, in response to receiving a command as described above with reference to block 805 of method 800, CPU core 108 may determine a programming order for one or more CPU instructions to be executed based on the command.
[0079] At block 910, instructions in one or more CPU instructions in a programmed order may be fetched and decoded. For example, in some embodiments, at block 910 of method 900, to execute one or more CPU instructions in a programmed order, the CPU instructions may be sequentially executed in a programmed order such that each of the instructions may be sequentially fetched, decoded, and dispatched (e.g., sent to an appropriate data path in a data path of the CPU pipeline 300). Thus, the instruction referred to by block 910 may be any one of the CPU instructions in a programmed order that is currently being fetched and decoded for dispatch by the CPU core 108.
[0080] At block 915, a determination may be made as to whether the instruction fetched and decoded at block 910 includes an acceleration workload encoded in the instruction. For example, in some embodiments, the CPU core 108 may determine whether the decoded instruction includes an acceleration workload from the instruction type of the decoded instruction. Based on the determination at block 915, the CPU core 108 may dispatch the decoded instruction to an appropriate data path in the data path of the CPU pipeline 300. For example, if the decoded instruction does not include an acceleration workload (e.g., no at block 915), the decoded instruction may be dispatched to an appropriate data path in the CPU data path 302 (e.g., scalar, memory, etc.). On the other hand, if the decoded instruction includes an acceleration workload (e.g., yes at block 915), the decoded instruction may be dispatched to the acceleration data path 304 (e.g., to one or more appropriate accelerator units of the accelerator 114).
[0081] Thus, in some embodiments, in response to determining that the decoded instruction does not include an accelerated workload (e.g., no at block 915), the decoded instruction may be dispatched to one or more of the CPU data paths at block 920, and method 900 may continue at block 935, which is described in more detail below (e.g., to determine whether the next instruction in the one or more CPU instructions in the programmed order includes an accelerated workload).
[0082] On the other hand, in some embodiments, in response to determining that the decoded instruction contains an acceleration workload (e.g., yes at block 915), the instruction (e.g., the decoded instruction) may be enqueued in the AIQ at block 925, and at block 930, a corresponding acceleration operand may be provided. For example, the decoded instruction may be dispatched to the accelerator 114 via the AIQ, and when the decoded instruction is dequeued from the AIQ. The corresponding acceleration operand of the decoded instruction may be provided to the accelerator 114 for calculation. Fig.10 The operation of the accelerator 114 based on the operations of blocks 925 and 930 of the method 900 is described in further detail.
[0083] Still refer to Fig. 9 , after the decoded instruction is dispatched (e.g., dispatched to the CPU data path at block 920 or to the accelerator data path at block 925), the method 900 may continue at block 935 to determine whether there are more instructions in the programming order. If so (e.g., yes at block 935), the method 900 may repeat at block 910 so that the next instruction in the programming order is fetched, decoded, and dispatched as described above. On the other hand, if there are no more instructions in the programming order (e.g., no at block 935), the method 900 may end. In this case, if any of the instructions in the programming order are dispatched to the accelerator 114 in the method 900, the CPU core 108 may then retrieve (e.g., read or be provided) the result of the accelerator 114 from the accelerator storage at block 825 of the method 800, and the result may be stored in the CPU memory (e.g., the shared memory 106) at block 830 of the method 800 as described above.
[0084] Fig.10 A flowchart of a method for processing an accelerated workload from an accelerated instruction queue according to one or more embodiments of the present disclosure is shown. However, the present disclosure is not limited to Fig.10 The order or number of operations of the method 1000 shown in the figure is not limited to the order or number of operations, and can be changed to any desired order or number of operations as recognized by those of ordinary skill in the art. For example, in some embodiments, the order thereof may be changed, some of the processes thereof may be performed simultaneously or sequentially, or the method 1000 may include fewer or additional operations.
[0085] Reference Fig.10, method 1000 may begin, and at block 1005, an instruction (e.g., a CPU instruction) may be dequeued from the AIQ, and at block 1010, a corresponding operand may be retrieved (e.g., read or provided). For example, in some embodiments, accelerator 114 (or a resource management agent in the accelerator) may dequeue instructions from the AIQ in the order in which they are received (e.g., a FIFO), and the operand may be data or a value stored in SCM 110 that corresponds to the dequeued instruction and is read by or provided to accelerator 116 to compute an accelerated workload encoded in the dequeued instruction (e.g., a CPU instruction).
[0086] In block 1015, intermediate outputs may be calculated based on the instruction and the corresponding operands, and in block 1020, final results based on the intermediate outputs may be stored in the accelerator memory 116. For example, in some embodiments, the processing elements PE of the accelerator 114 may calculate intermediate outputs during multiple periods or multiple stages of the accelerator data path, and a collection of all final outputs of each of the processing elements PE collected during the multiple periods or multiple stages may correspond to the final result. In some embodiments, the intermediate outputs and the final outputs of each of the processing elements PE calculated during the multiple periods or multiple stages may first be temporarily stored in the accumulation buffer ACC until the final result is calculated, and in the last stage of the accelerator data path, in block 1020, the final result may be submitted to the accelerator memory (e.g., to a register file in the accelerator memory).
[0087] At block 1025, a determination may be made as to whether there are more instructions queued in the AIQ. If so (e.g., yes at block 1025), then as described above, method 1000 may repeat from block 1005 until all instructions in the AIQ are sequentially dequeued, computed, and stored (e.g., one at a time). On the other hand, if there are no instructions queued in the AIQ (e.g., no at block 1025), then method 1000 may end. As described above with reference to Figure 8 As described in blocks 825 and 830 of method 800, in some embodiments, the CPU core 108 may retrieve results (e.g., final results) from the accelerator memory 114 when appropriate (e.g., after multiple periods or multiple stages of the accelerator data path are completed), and may store the results in a CPU memory (e.g., shared memory 106) for access by the main CPU 102 (e.g., a requesting application running on the main CPU 102).
[0088] According to one or more embodiments of the present disclosure as described above, a neural network accelerator based on CPU instructions may be provided (e.g., such as for model training and / or inference) to improve computing performance, programmability, and power efficiency. However, the present disclosure is not limited thereto, and additional aspects and additional features may be clear from the above embodiments, or may be learned by practicing one or more of the presented embodiments of the present disclosure.
[0089] The foregoing is an illustration of some embodiments of the present disclosure and should not be construed as limiting the present disclosure. When a certain embodiment can be implemented differently, the specific processing order may be different from the order described. For example, two consecutively described processes may be performed simultaneously or substantially simultaneously, or may be performed in an order opposite to the order described.
[0090] For clarity, in the accompanying drawings, the relative sizes, thicknesses, and ratios of elements, layers, and regions may be exaggerated and / or simplified. For ease of explanation, spatially relative terms may be used herein, (such as, "below," "below," "lower," "below," "above," "upper," etc.) to describe the relationship between an element or a feature and another element or another feature as shown in the accompanying drawings. It will be understood that, in addition to the directions shown in the accompanying drawings, spatially relative terms are intended to cover different directions of the device in use or in operation. For example, if the device in the accompanying drawings is flipped, the element described as "below" or "below" or "below" other elements or other features will be oriented to be "above" other elements or other features. Therefore, the example terms "below" and "below" may cover the directions above and below. The device may be oriented in other ways (e.g., rotated 90 degrees or in other directions), and the spatially relative descriptors used herein should be interpreted accordingly.
[0091] It will be understood that although the terms "first", "second", "third" and the like may be used herein to describe various elements, components, regions, layers and / or parts, these elements, components, regions, layers and / or parts are not limited by these terms. These terms are used to distinguish one element, component, region, layer or part from another element, component, region, layer or part. Therefore, without departing from the spirit and scope of the present disclosure, the first element, first component, first region, first layer or first part described above may be referred to as a second element, second component, second region, second layer or second part.
[0092] It will be understood that when an element or layer is referred to as being "on," "connected to," or "bound to" another element or layer, it can be directly on, directly connected to, or bound to the other element or layer, or there may be one or more intervening elements or layers. Similarly, when a layer, region, or element is referred to as being "electrically connected to" another layer, region, or element, it can be directly electrically connected to the other layer, region, or element, and / or can be indirectly electrically connected to one or more intervening layers, regions, or elements therebetween. Furthermore, it will be understood that when an element or layer is referred to as being "between" two elements or layers, it can be the only element or layer between the two elements or layers, or there may also be one or more intervening elements or layers.
[0093] The terms used herein are for the purpose of describing a particular embodiment, and are not intended to limit the present disclosure. As used herein, unless the context clearly states otherwise, the singular form is intended to also include the plural form. It will also be understood that when used in this specification, the terms "include", "comprise", and "have" specify the features, wholes, steps, operations, elements and / or components that are described, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their groups. As used herein, the term "and / or" includes any combination and all combinations of one or more of the associated listed items. For example, the statement "A and / or B" means A, B or A and B. Statements such as "at least one of ...", when after the element list, modify the entire column of elements, rather than modifying the individual elements in the column. For example, the statements "at least one of a, b or c", "at least one of a, b and c" and "at least one selected from the group consisting of a, b and c" indicate only a, only b, only c, both a and b, both a and c, both b and c, all a, b and c, or its deformation.
[0094] As used herein, the terms "substantially," "about," and similar terms are used as terms of approximation rather than terms of degree, and are intended to take into account the inherent variation in measurements or calculations that one of ordinary skill in the art would recognize. Additionally, when describing embodiments of the present disclosure, the use of "may" means "one or more embodiments of the present disclosure." As used herein, the term "using" may be considered synonymous with the term "utilizing."
[0095] The electronic device or electrical device and / or any other related device or related component (e.g., main CPU, CPU core, accelerator, various units of accelerator, etc.) according to the embodiments of the present disclosure described herein can be implemented using any suitable hardware, firmware (e.g., application specific integrated circuit), software, or a combination of software, firmware, and hardware. For example, the various components of these devices can be formed on an integrated circuit (IC) chip or on a separate IC chip. In addition, the various components of these devices can be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on a substrate. In addition, the various components of these devices can be processes or threads running on one or more processors in one or more computing devices, and one or more processors execute computer program instructions and interact with other system components for performing the various functions described herein. Computer program instructions are stored in a memory that can be implemented in a computing device using a standard memory device (such as, for example, a random access memory (RAM)). Computer program instructions can also be stored in other non-temporary computer-readable media (such as, for example, CD-ROMs, flash drives, etc.). In addition, those skilled in the art should recognize that the functionality of various computing devices may be combined or integrated into a single computing device, or the functionality of a particular computing device may be distributed across one or more other computing devices without departing from the spirit and scope of the example embodiments of the present disclosure.
[0096] Although some embodiments have been described, it will be readily understood by those skilled in the art that various modifications may be made in the embodiments without departing from the spirit and scope of the present disclosure. It will be understood that, unless otherwise stated, the description of the features or aspects in each embodiment should generally be considered to be applicable to other similar features or similar aspects in other embodiments. Therefore, it will be clear to those of ordinary skill in the art that, unless otherwise specifically stated, the features, characteristics and / or elements described in conjunction with a particular embodiment may be used alone or in combination with the features, characteristics and / or elements described in conjunction with other embodiments. Therefore, it will be understood that the foregoing is an illustration of various example embodiments and should not be construed as being limited to the specific embodiments disclosed herein, and that various modifications to the disclosed embodiments and other example embodiments are intended to be included in the spirit and scope of the present disclosure as defined in the appended claims and their equivalents.
Claims
1. An integrated circuit, include: Central processing unit CPU core; accelerator; as well as Acceleration instruction queue, connected to CPU core and accelerator, Among them, the CPU core is configured as: fetching and decoding one or more instructions from a sequence of instructions in a programmed order; determining an instruction from among the one or more instructions that includes an acceleration workload, wherein the acceleration workload is encoded in the instruction; and The instructions containing the acceleration workload are queued in an acceleration instruction queue.
2. The integrated circuit according to claim 1, in, The accelerator is configured as: dequeueing the instruction containing the acceleration workload from the acceleration instruction queue; receiving operands associated with the accelerated workload from a temporary memory of the CPU core; and A result is computed based on the operands and the instruction that was dequeued.
3. The integrated circuit according to claim 2, in, The accelerator is configured to dequeue instructions from the acceleration instruction queue in a first-in-first-out method.
4. The integrated circuit according to claim 2, in, The accelerator is further configured to store the result in an embedded memory of the accelerator.
5. The integrated circuit according to claim 4, in, The CPU core, accelerator, temporary memory and embedded memory are integrated with each other on the same chip.
6. The integrated circuit according to claim 4, in, The CPU core is configured to retrieve the result from the embedded memory of the accelerator and store the result in a temporary memory of the CPU core.
7. The integrated circuit according to claim 1, in, The accelerator instruction queue includes a plurality of instruction queues having different priorities defined for the accelerator.
8. A computing system, include: accelerator; One or more processors integrated with the accelerator in the same integrated circuit; as well as a memory, comprising instructions stored on the memory, which instructions, when executed by the one or more processors, cause the one or more processors to: identifying a programmed sequence for executing one or more central processing unit (CPU) instructions; and executing said one or more CPU instructions according to said programmed sequence, Wherein, to execute the one or more CPU instructions, the instructions cause the one or more processors to: fetching and decoding a first instruction from among the one or more CPU instructions in the programmed order; and The decoded first instruction is dispatched to a CPU data path or an accelerator data path in a CPU pipeline based on an instruction type of the first instruction.
9. The computing system according to claim 8, in, The first instruction includes an acceleration workload to be dispatched to the accelerator data path, wherein the acceleration workload is encoded in the first instruction, and The instructions further cause the one or more processors to: Enqueue the first instruction in the acceleration instruction queue; and A corresponding operand is provided to an accelerator for calculation based on the first instruction.
10. The computing system according to claim 9, in, The accelerator is configured as: Dequeue the first instruction from the acceleration instruction queue; calculating a result based on the corresponding operand and a first instruction dequeued from the accelerated instruction queue; and The results are stored in an embedded memory of the accelerator.
11. The computing system according to claim 10, in, The accelerator is configured to dequeue the first instruction from the acceleration instruction queue in a first-in-first-out method.
12. The computing system according to claim 10, in, The instructions further cause the one or more processors to: retrieving the result from an embedded memory of the accelerator; and The results are stored in temporary memory.
13. The computing system according to claim 12, in, The accelerator, the one or more processors, the embedded memory and the temporary storage are integrated into the same integrated circuit.
14. The computing system according to claim 8, in, A first instruction is dispatched to a CPU datapath, and the instruction further causes the one or more processors to: extracting and decoding a second instruction from among the one or more CPU instructions in the programmed order; determining an accelerated workload encoded in the second instruction; Enqueue the second instruction in the acceleration instruction queue; and The corresponding operand is provided to the accelerator for calculation based on the second instruction.
15. A method for accelerating an instruction, include: recognizing, by one or more processors, a programmed order for executing the one or more instructions; determining, by the one or more processors in the programmed order, an acceleration workload encoded in instructions of the one or more instructions; as well as The instruction is dispatched, by the one or more processors, to an accelerator datapath among a plurality of datapaths of a CPU pipeline based on determining that an acceleration workload is encoded in the instruction.
16. The method according to claim 15, in, The steps of dispatching include: Enqueuing, by the one or more processors, the instruction in an accelerated instruction queue; and The one or more processors provide corresponding operands to an accelerator data path for calculation based on the instruction.
17. The method according to claim 16, in, The accelerator data path includes an accelerator integrated in a same integrated circuit as the one or more processors, and the method further includes: The accelerator dequeues the instruction from the acceleration instruction queue; calculating, by the accelerator, a result based on the corresponding operand and the instruction dequeued from the acceleration instruction queue; and The result is stored by the accelerator in an embedded memory of the accelerator.
18. The method according to claim 17, in, The accelerator is configured to dequeue instructions from the acceleration instruction queue in a first-in-first-out method.
19. The method according to claim 17, further comprising: include: retrieving, by the one or more processors, the result from an embedded memory of the accelerator; as well as The results are stored by the one or more processors in a temporary memory of the one or more processors.
20. The method according to claim 17, in, The accelerator and the one or more processors are co-processors or multi-processors of the same integrated circuit.
