A fusion processor and a fusion processing method
By designing a fusion processor, which adopts a shared execution unit between the central processing unit and the graphics processing unit, the problem of wasted computing resources caused by different instruction sets is solved, and efficient computing power fusion is achieved to adapt to different computing needs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHIHEXINGYI TECHNOLOGY (SHANGHAI) CO LTD
- Filing Date
- 2026-01-22
- Publication Date
- 2026-06-19
AI Technical Summary
Despite their different instruction sets, existing general-purpose processors and graphics processors struggle to efficiently integrate their respective computing architectures and data structures when faced with the ever-expanding demands of AI computing, leading to wasted computing resources and performance bottlenecks.
Design a fusion processor that includes a central processing unit front-end and a graphics processing unit front-end, sharing an execution unit. It executes the instruction sets of the central processing unit and the graphics processing unit respectively through alternating time cycles, thereby achieving parallel processing of instructions and resource sharing.
It achieves dual processing capabilities of general-purpose computing and parallel computing, improving computing efficiency, reducing resource waste, and adapting to scenarios with different computing needs.
Smart Images
Figure CN121560577B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of processor technology, and in particular to a fusion processor and a fusion processing method. Background Technology
[0002] The mainstream instruction sets for general-purpose processors (CPUs) are currently x86, ARM, and RISC-V. While their addressing modes and coding styles differ, their overall instruction set structure and supported operations are largely the same. Graphics processing units (GPUs) mostly use closed-source instruction sets, with PTX as the intermediate instruction set. AMD has open-sourced its underlying instruction set. Comparing the two, aside from some minor differences such as branch scheduling and asynchronous execution, their overall instruction set structure and functionality are converging.
[0003] Despite their different instruction sets, general-purpose processors and graphics processors are constantly evolving their own computing architectures and data structures as computing demands, particularly those driven by AI, continue to grow. Summary of the Invention
[0004] In view of this, the present disclosure provides a fusion processor and a fusion processing method.
[0005] According to a first aspect of the present disclosure, a fusion processor is provided, comprising: a central processing unit front-end for fetching, decoding, and scheduling a central processing unit instruction set; a graphics processing unit front-end for fetching, decoding, and scheduling a graphics processing unit instruction set; and a shared execution unit for executing the central processing unit instructions driven by the central processing unit front-end in a first time period and executing the graphics processing unit instruction set driven by the graphics processing unit front-end in a second time period, wherein the first time period is different from the second time period.
[0006] According to a second aspect of the present disclosure, a fusion processing method is provided, comprising: fetching, decoding, and scheduling a central processing unit instruction set; fetching, decoding, and scheduling a graphics processing unit instruction set; executing the central processing unit instructions by a shared execution unit driven by the central processing unit front-end in a first time period; and executing the graphics processing unit instruction set by a shared execution unit driven by the graphics processing unit front-end in a second time period, wherein the first time period is different from the second time period.
[0007] According to the fusion processor scheme of this disclosure, its hardware architecture includes two front-ends: a central processing unit (CPU) front-end and a graphics processing unit (GPU) front-end. The CPU front-end, designed to meet the needs of general-purpose computing, can support complex branch prediction, high-bandwidth instruction fetching, and superscalar out-of-order decoding and scheduling. The GPU front-end is designed with multiple simple instruction fetching and decoding operations, and scheduling to support hardware multithreading. In this disclosure, the shared execution unit is driven by the CPU front-end in the first time period and by the GPU front-end in the second time period. This disclosure provides both general-purpose computing capabilities and parallel computing processing capabilities. Attached Figure Description
[0008] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings.
[0009] Figure 1 This is a schematic diagram of the structure of a fusion processor according to an embodiment of the present disclosure;
[0010] Figure 2 This is a schematic diagram of a typical central processing unit (CPU).
[0011] Figure 3 This is a schematic diagram of a typical graphics processing unit (GPU).
[0012] Figure 4 This is a flowchart of a fusion processing method according to another embodiment of the present disclosure. Detailed Implementation
[0013] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0014] It should be noted that the headings of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and embodiments of any type may be included under any section / subsection. Furthermore, embodiments described in any section / subsection may be combined in any way with any other embodiments described in the same section / subsection and / or different sections / subsections.
[0015] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below. The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.
[0016] The embodiments of this disclosure may involve user data, data acquisition, and / or use. All of these aspects comply with applicable laws, regulations, and relevant provisions. In the embodiments of this disclosure, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing the embodiments of this disclosure, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained in accordance with relevant laws and regulations through appropriate means. The specific methods of notification and / or authorization may vary depending on the actual situation and application scenario, and the scope of this disclosure is not limited in this respect.
[0017] In this specification and the embodiments, any processing of personal information will be carried out only under the premise of legality (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.
[0018] The fusion processor solution provided in the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.
[0019] See Figure 1 This disclosure provides a fusion processor, which includes:
[0020] A central processing unit front-end 11 is used for fetching, decoding, and scheduling the central processing unit instruction set.
[0021] A graphics processing unit front-end 12 is used for fetching, decoding, and scheduling the graphics processing unit instruction set.
[0022] The shared execution unit 13 is used to execute CPU instructions in a first time period driven by the CPU front end and to execute the GPU instruction set in a second time period driven by the GPU front end, wherein the first time period is different from the second time period.
[0023] In some specific implementations of the embodiments of this disclosure, the central processing unit front-end 11 and the graphics processing unit front-end 12 read register files through their respective memory access subunits and send them to the shared execution unit 13.
[0024] Central processing units (CPUs) are designed with latency in mind, prioritizing instruction-level parallelism. See also... Figure 2 The hardware architecture of a central processing unit (CPU) consists of several major components, including the instruction fetch unit, decoding unit, scheduling unit, execution unit, and memory access unit. The instruction fetch unit, decoding unit, scheduling unit, and memory access unit are usually referred to as the CPU front-end.
[0025] The CPU front-end (Front-end) unit of the CPU is responsible for instruction fetching and branch processing, typically employing complex branch prediction techniques to improve fetch bandwidth. The CPU front-end's decoding unit is responsible for instruction decoding and register dependency detection. The CPU front-end's scheduling unit is responsible for instruction issuance, register file access, and data forwarding to accelerate operand acquisition, typically using out-of-order multiple issuance and physical register renaming techniques to improve parallel instruction issuance and maximize instruction parallelism within a single thread. The CPU front-end's memory access unit is responsible for executing memory access instructions, typically utilizing the principle of data locality to implement multi-level caches and data prefetching to accelerate data retrieval performance.
[0026] Graphics processors are designed with throughput in mind, employing hardware multithreading technology to achieve thread-level parallelism. See also Figure 3 The hardware architecture of a graphics processing unit (GPU) also includes an instruction fetch unit, a decoding unit, a scheduling unit, an execution unit, and a memory access unit. The instruction fetch unit, decoding unit, scheduling unit, and memory access unit are usually referred to as the GPU front-end 12.
[0027] In this architecture, the graphics processing unit (GPU) front-end 12 shares a single fetch unit, decode unit, and scheduler unit within the same thread bundle, while multiple thread bundles have different fetch units, decode units, and schedulers. Multiple thread bundles share a single execution unit and memory access unit, with execution and memory access units being time-multiplexed among the thread bundles to improve throughput. Therefore, each fetch unit, decode unit, and memory access unit of the GPU is very simple, without complex branch prediction or the need for various out-of-order superscalar optimization techniques. The memory access unit of the GPU front-end 12 typically consists of a cache and local memory, supporting multi-address merging operations and providing parallelism for memory access.
[0028] The fusion processor 1 of this embodiment can run both the central processing unit instruction set and the graphics processing unit instruction set, and the central processing unit front-end 11 and the graphics processing unit front-end 12 share the shared execution unit 13.
[0029] The hardware architecture of the fusion processor 1 in this embodiment includes two front-ends: a central processing unit (CPU) front-end 11 and a graphics processing unit (GPU) front-end 12. The CPU front-end 11, designed to support complex branch prediction, high-bandwidth instruction fetching, and superscalar out-of-order decoding and scheduling, is tailored to the needs of general-purpose computing. The GPU front-end 12 is designed with multiple simple instruction fetching and decoding operations, and scheduling to support hardware multithreading. The shared execution unit 13 of the fusion processor 1 in this embodiment can be driven by either the CPU front-end 11 or the GPU front-end 12. This embodiment possesses both general-purpose computing capabilities and parallel computing capabilities.
[0030] Specifically, in the operating mode of the fusion processor 1 in this embodiment, the central processing unit front-end 11 can independently and in parallel perform instruction fetching, decoding, scheduling, and memory access in the first time period, while the graphics processing unit front-end 12 does not perform instruction fetching, decoding, scheduling, and memory access; the graphics processing unit front-end 12 can independently and in parallel perform instruction fetching, decoding, scheduling, and memory access in the second time period, while the central processing unit front-end 11 does not perform instruction fetching, decoding, scheduling, and memory access. That is, the central processing unit front-end 11 and the graphics processing unit front-end 12 share the execution unit 13 in a time-sharing manner.
[0031] The time-sharing multiplexing of the shared execution unit 13 can be performed in a fine-grained or coarse-grained manner.
[0032] Fine-grained time-sharing multiplexing means that the CPU front-end 11 and the GPU front-end 12 run simultaneously, each of which can send instructions to the shared execution unit 13. The shared execution unit 13 receives instructions from the CPU front-end 11 and the GPU front-end 12 according to the idle level of each sub-unit in the shared execution unit 13.
[0033] Coarse-grained time-division multiplexing refers to receiving instructions from only the CPU front-end 11 or the GPU front-end 12 within the same time period. In other words, only instructions from one front-end are received in a phase, while the other front-end stops sending instructions. In this case, one of the front-ends can enter standby or low-power mode to reduce power consumption.
[0034] In this embodiment of the present disclosure, the fusion processor 1 performs time-sharing multiplexing of the shared execution unit 13 in a fine-grained or coarse-grained manner, which can set the time-sharing multiplexing of the shared execution unit 13 according to different application scenarios to meet different needs.
[0035] Specifically, coarse-grained time-division multiplexing is typically used in power-sensitive applications or applications with a single task type. Coarse-grained time-division multiplexing can reduce power consumption, achieve maximum energy efficiency, and extend the device's uptime.
[0036] Fine-grained time-sharing multiplexing is typically used in scenarios where high throughput, low latency, or highly mixed task loads are required. Fine-grained time-sharing multiplexing reduces the idle waiting time of execution units, maximizing the utilization of shared hardware resources and the overall instruction throughput of the system.
[0037] In some specific implementations of the embodiments of this disclosure, the shared execution unit 13 includes:
[0038] One or more of the scalar execution subunit, vector execution subunit, and matrix execution subunit.
[0039] In this embodiment of the fusion processor 1, the central processing unit front-end 11 and the graphics processing unit front-end 12 can share one or more of the scalar execution subunit, vector execution subunit, and matrix execution subunit, while the non-shared execution units are kept independent by the central processing unit front-end 11 or the graphics processing unit front-end 12.
[0040] The embodiments disclosed herein can more flexibly select one or more of the shared scalar execution subunit, vector execution subunit, and matrix execution subunit according to the needs of the algorithms executed by the central processing unit front-end 11 and the graphics processing unit front-end 12, thereby enabling more flexible configuration.
[0041] Specifically, the scalar execution subunit is used to execute scalar instructions of the central processing unit front-end 11 or the graphics processing unit front-end 12.
[0042] Among them, the scalar instructions of the central processing unit front end include: scalar integer and scalar floating point.
[0043] Specifically, scalar integer data types include one or more of INT8, INT16, INT32, and INT64, while floating-point data types include one or more of FP16, BF16, FP32, and FP64. Integer operations mainly include addition, subtraction, multiplication, multiply-accumulate, division, comparison, logical operations, and jump operations, while scalar floating-point operations mainly include addition, multiplication, multiply-accumulate, division / expansion, and comparison operations.
[0044] Scalar instructions in the graphics processor front end include: the same data executed by multiple threads.
[0045] Specifically, in this embodiment of the disclosure, the same data executed by 32 threads is extracted and used as scalar instructions for the graphics processor front end.
[0046] The vector execution subunit is used to execute vector instructions from the central processing unit front-end 11 or the graphics processing unit front-end 12.
[0047] The vector instructions of the central processing unit front end include:
[0048] A single instruction that supports the parallel execution of multiple integers or floating-point numbers.
[0049] Vector instructions in the front-end of the central processing unit support the parallel execution of multiple integers or floating-point numbers in a single instruction, and the operations are basically the same as the scalar operations mentioned above.
[0050] The vector instructions in the front end of the central processing unit contain 32 vector registers. RISC-V defines the bit width of the vector bit width indicator register. The vector bit width can be 128, 256, 512, 1024 bits or even larger, depending on the hardware implementation.
[0051] Vector instructions for the graphics processing unit (GPU) front end include:
[0052] A single thread bundle consists of multiple threads, and the single thread bundle uses the same instructions to process different data.
[0053] The graphics processing unit (GPU) uses a single instruction multiple thread (SIMT) architecture, which combines 32 threads into a thread bundle. The thread bundle uses the same instructions to process different data, which is essentially the same as single instruction multiple data and can be regarded as vector instructions.
[0054] The matrix execution subunit is used to execute matrix instructions from the central processing unit front-end 11 or the graphics processing unit front-end 12.
[0055] Specifically, the matrix execution subunit is responsible for executing matrix multiplication instructions. The matrix execution subunit includes a multiply-accumulate unit. Within the matrix execution subunit, numerous multiply-accumulate units work in parallel, completing multiple multiply-accumulate operations at once, thus efficiently implementing matrix multiplication instructions.
[0056] The matrix instructions of the central processing unit front end include:
[0057] Performing three-dimensional matrix multiplication on a two-dimensional matrix.
[0058] The matrix instructions of the central processing unit front end support data types including FP8, FP16, BF16, INT8, etc.
[0059] The matrix instructions of the graphics processing unit front end include:
[0060] Instructions responsible for 2D matrix multiplication operations.
[0061] This disclosure divides both the graphics processing unit (GPU) instruction set and the central processing unit (CPU) instruction set into three parts: scalar instructions, vector instructions, and matrix instructions. The operation types and data types supported by each type of instruction are essentially the same for both the GPU and CPU instruction sets. This disclosure employs a converged computing architecture in its converged processor, which possesses both general-purpose computing capabilities and parallel computing capabilities.
[0062] In some specific implementations of this disclosure, the first time period ends and the second time period begins, and the second time period ends and the first time period begins.
[0063] In fine-grained time-sharing multiplexing, the central processing unit front-end 11 and the graphics processing unit front-end 12 of this embodiment issue instructions in parallel. The first time period and the second time period alternate. The first time period executes the instructions sent by one of the front-ends, and the second time period executes the instructions sent by the other front-end. This time period can be as short as 1 clock cycle, thereby improving the utilization rate of the shared execution unit 13 and avoiding resource waste caused by the idleness of the shared execution unit 13.
[0064] In coarse-grained time-sharing multiplexing, in some specific implementations of this disclosure, only one of the central processing unit front-end 11 and the graphics processing unit front-end 12 works. After the central processing unit front-end 11 finishes the first time period, the graphics processing unit front-end 12 starts running in the second time period. This time period is usually a time slice, for example, at the millisecond level.
[0065] Specifically, the first jump instruction and the second jump instruction can be the same or different. The embodiments of this disclosure use the first jump instruction and the second jump instruction to switch between the first time period and the second time period, which is convenient and does not require complex design.
[0066] See Figure 4 This disclosure also provides a fusion processing method, including:
[0067] Step S1: Fetch, decode, and schedule the instruction set of the central processing unit.
[0068] Step S2: Fetch, decode, and schedule the graphics processor instruction set.
[0069] Step S3: In the first time period, the shared execution unit driven by the central processing unit front end executes the central processing unit instructions. In the second time period, the shared execution unit driven by the graphics processing unit front end executes the graphics processing unit instruction set. The first time period is different from the second time period.
[0070] According to the fusion processor scheme of this disclosure, its hardware architecture includes two front-ends: a central processing unit (CPU) front-end and a graphics processing unit (GPU) front-end. The CPU front-end, designed to meet the needs of general-purpose computing, can support complex branch prediction, high-bandwidth instruction fetching, and superscalar out-of-order decoding and scheduling. The GPU front-end is designed with multiple simple instruction fetching and decoding operations, and scheduling to support hardware multithreading. In this disclosure, the shared execution unit is driven by the CPU front-end in the first time period and by the GPU front-end in the second time period. This disclosure provides both general-purpose computing capabilities and parallel computing processing capabilities.
[0071] It should be understood that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for the method embodiments, since they are basically similar to the methods described in the apparatus and system embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions of other embodiments.
[0072] It should be understood that the foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0073] It should be understood that the use of a singular form to describe an element or to show only one element in the accompanying drawings does not imply that the number of such element is limited to one. Furthermore, modules or elements described or shown as separate herein may be combined into a single module or element, and modules or elements described or shown as single herein may be broken down into multiple modules or elements.
[0074] It should also be understood that the terminology and expressions used herein are for descriptive purposes only, and one or more embodiments described herein should not be limited to these terms and expressions. The use of these terms and expressions does not exclude any illustrative and descriptive equivalent features (or parts thereof), and it should be recognized that various modifications that may exist should also be included within the scope of the claims. Other modifications, variations, and substitutions may also exist. Accordingly, the claims should be considered to cover all such equivalents.
Claims
1. A fusion processor, characterized in that, The fusion processor includes: A central processing unit front end, used for fetching, decoding, and scheduling the central processing unit instruction set; A graphics processing unit (GPU) front end is used for fetching, decoding, and scheduling the GPU instruction set; A shared execution unit is configured to execute CPU instructions driven by the CPU front-end in a first time period and execute the GPU instruction set driven by the GPU front-end in a second time period, wherein the first time period is different from the second time period; the end of the first time period and the start of the second time period are triggered by a first jump instruction. The scalar instructions of the graphics processor front end include: the same data executed by multiple threads extracted from the processor. The vector instructions of the graphics processor front end include: A single thread bundle consisting of multiple threads, wherein the single thread bundle uses the same instructions to process different data.
2. The fusion processor according to claim 1, characterized in that, The shared execution unit includes: One or more of the scalar execution subunit, vector execution subunit, and matrix execution subunit.
3. The fusion processor according to claim 2, characterized in that, The scalar execution subunit is used to execute scalar instructions from the front end of the central processing unit or the front end of the graphics processing unit. The vector execution subunit is used to execute vector instructions from the front end of the central processing unit or the front end of the graphics processing unit; The matrix execution subunit is used to execute matrix instructions from the front end of the central processing unit or the front end of the graphics processing unit.
4. The fusion processor according to claim 3, characterized in that, The scalar instructions of the central processing unit front end include: scalar integer and scalar floating point.
5. The fusion processor according to claim 3, characterized in that, The vector instructions of the central processing unit front end include: A single instruction that supports the parallel execution of multiple integers or floating-point numbers.
6. The fusion processor according to claim 3, characterized in that, The matrix instructions of the central processing unit front end include: Performing three-dimensional matrix multiplication on two-dimensional matrices; The matrix instructions of the graphics processor front end include: Instructions responsible for 2D matrix multiplication operations.
7. The fusion processor according to claim 1, characterized in that, The first time period ends and the second time period begins; the second time period ends and the first time period begins.
8. The fusion processor according to claim 7, characterized in that, The end of the second time period and the start of the first time period are triggered by a second jump instruction.
9. The fusion processor according to claim 1, characterized in that, The central processing unit front end and the graphics processing unit front end read register files through their respective memory access subunits and send them to the shared execution unit.
10. A fusion processing method, characterized in that, include: Fetch, decode, and schedule the instruction set of the central processing unit; Fetch, decode, and schedule the instruction set of the graphics processor; In a first time period, the shared execution unit driven by the central processing unit front end executes the central processing unit instructions; in a second time period, the shared execution unit driven by the graphics processing unit front end executes the graphics processing unit instruction set. The first time period is different from the second time period. The end of the first time period and the start of the second time period are triggered by a first jump instruction. The scalar instructions of the graphics processor front end include: the same data executed by multiple threads extracted from the processor. The vector instructions of the graphics processor front end include: A single thread bundle consisting of multiple threads, wherein the single thread bundle uses the same instructions to process different data.