Instruction processing device, processor, computer device, server cluster and method

Abstract

The application discloses an instruction processing device, a processor, a computer device, a server cluster and a method. The device comprises a storage unit and an execution unit; the storage unit is connected with the execution unit; the storage unit comprises a first storage unit and a second storage unit; the first storage unit is configured to store source operands of instructions in a single instruction multiple thread (SIMT) mode or store source operands of instructions in a single instruction multiple data (SIMD) mode; the second storage unit is configured to store source operands of instructions in the SIMT mode; and the execution unit is configured to perform calculation based on the source operands stored in at least one of the first storage unit and the second storage unit in the case of running in the SIMT mode, and perform calculation based on the source operands stored in the first storage unit in the case of running in the SIMD mode. The device can be compatible with the SIMT and the SIMD architecture, reduce deployment cost and improve resource utilization.

Description

Technical Field

[0001] This application relates to the field of processors, and in particular to an instruction processing device, processor, computer equipment, server cluster, and method. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology, AI processors need to efficiently handle large-scale parallel computing tasks. Single Instruction Multiple Data (SIMD) and Single Instruction Multiple Threads (SIMT) are two mainstream parallel computing architectures, significantly improving computing performance through data-level parallelism and thread-level parallelism, respectively. The SIMD architecture processes multiple data elements synchronously with a single instruction, making it suitable for rule-based computations (such as matrix operations); while the SIMT architecture executes the same instruction in parallel through multiple threads, supporting complex branching logic (such as graphics rendering and dynamic neural networks).

[0003] In related technologies, AI processors typically support only a single architecture (SIMD or SIMT), while operators in AI workloads may be implemented using different architectures due to historical reasons or performance requirements. For example, early operators were developed based on the SIMD architecture, while newly optimized operators may be more suitable for the SIMT architecture. Because of the significant differences in hardware design and software ecosystem between the two architectures, two separate hardware sets are required to support hybrid operators, such as SIMD accelerators and SIMT graphics processing units (GPUs), resulting in high deployment costs and low resource utilization. Summary of the Invention

[0004] This application provides an instruction processing device, a processor, a computer device, a server cluster, and a method. The technical solution is as follows.

[0005] On one hand, an instruction processing apparatus is provided, the instruction processing apparatus including a storage unit and an execution unit; the storage unit is connected to the execution unit; the storage unit includes a first storage unit and a second storage unit;

[0006] The first storage unit is configured to store the source operands of instructions in Single Instruction Multiple Thread (SIMT) mode, or to store the source operands of instructions in Single Instruction Multiple Data (SIMD) mode.

[0007] The second storage unit is configured to store the source operands of the instructions in the SIMT mode;

[0008] The execution unit is configured to perform computations based on source operands stored in at least one of the first and second storage units when operating in the SIMT mode; and to perform computations based on source operands stored in the first storage unit when operating in the SIMD mode.

[0009] In some embodiments, the apparatus further includes a reading unit connected to both the storage unit and the execution unit;

[0010] The read unit is configured to, when operating in the SIMT mode, read the source operand of the instruction from at least one of the first and second storage units; and send the source operand to the execution unit to perform the calculation.

[0011] The read unit is further configured to, when operating in the SIMD mode, read the source operand of the instruction from the first storage unit and send the source operand to the execution unit to perform the calculation.

[0012] In some embodiments, the read unit is further configured to read the source operand of the instruction from the first storage unit when operating in the SIMT mode; and to read the source operand of the instruction from the second storage unit when the source operand of the instruction is not included in the first storage unit.

[0013] The read unit is further configured to read the source operand of the instruction from the first storage unit when operating in the SIMD mode, wherein the second storage unit is disabled.

[0014] In some embodiments, the execution unit is further configured to write the calculation result back to the storage unit after performing a calculation based on the source operand of the instruction.

[0015] In some embodiments, the execution unit is further configured to write the calculation result back to the first storage unit when running in the SIMT mode; wherein, when the number of calculation results stored in the first storage unit reaches a preset threshold, the calculation result earliest written to the first storage unit is migrated to the second storage unit according to the first-in-first-out principle.

[0016] The execution unit is further configured to write the computation result back to the first storage unit when operating in the SIMD mode, wherein the second storage unit is disabled.

[0017] In some embodiments, the apparatus further includes an address generation unit, an instruction buffer unit, and an instruction issuing unit, wherein the address generation unit is connected to the instruction buffer unit, the instruction buffer unit is connected to the instruction issuing unit, and the instruction issuing unit is connected to the execution unit;

[0018] The address generation unit is configured to generate the program address of a thread, and the program address is used to indicate the storage location in memory of the instruction corresponding to the thread.

[0019] The instruction buffer unit is configured to temporarily store instructions to be issued based on the program address;

[0020] The instruction issuing unit is configured to issue instructions temporarily stored in the instruction buffer unit to the execution unit to perform calculations.

[0021] In some embodiments, the address generation unit includes a first generation unit and a second generation unit;

[0022] The first generation unit is configured to generate the program address for at least two threads in the SIMT mode respectively;

[0023] The second generation unit is configured to generate the program address for the first thread in the SIMD mode.

[0024] In some embodiments, the apparatus further includes a multiplexer and an instruction cache unit, wherein the input of the multiplexer is connected to the first generation unit and the second generation unit respectively, the output of the multiplexer is connected to the instruction cache unit, and the instruction cache unit is connected to the instruction buffer unit;

[0025] The multiplexer is configured to send the program address generated by the first generation unit to the instruction cache unit when operating in the SIMT mode; and to send the program address generated by the second generation unit to the instruction cache unit when operating in the SIMD mode.

[0026] The instruction cache unit is configured to retrieve the instruction based on the program address and temporarily store the instruction in the instruction buffer unit.

[0027] In some embodiments, the instruction buffer unit includes a first buffer unit and a second buffer unit. The first buffer unit includes at least two buffer queues, and the i-th buffer queue in the at least two buffer queues is used to temporarily store the instruction corresponding to the i-th thread in the at least two threads in the SIMT mode, where i is a positive integer. The second buffer unit includes a single queue, and the single queue is used to temporarily store the instruction corresponding to the first thread in the SIMD mode.

[0028] The instruction cache unit is further configured to temporarily store the instructions in the first buffer unit when operating in the SIMT mode;

[0029] The instruction cache unit is further configured to temporarily store the instructions in the second buffer unit when operating in the SIMD mode.

[0030] On the other hand, a processor is provided, the processor comprising: the instruction processing means as described above.

[0031] On the other hand, a computer device is provided, the computer device comprising: an instruction processing apparatus as described above, or a processor as described above.

[0032] Another approach provides a server cluster comprising at least two computer devices, the computer devices comprising: an instruction processing device as described above, or a processor as described above.

[0033] On the other hand, a method is provided that is applied to an instruction processing apparatus, the instruction processing apparatus including a storage unit and an execution unit; the instruction processing apparatus including a storage unit and an execution unit; the storage unit being connected to the execution unit; the storage unit including a first storage unit and a second storage unit;

[0034] The method includes:

[0035] The first storage unit stores the source operands of instructions in Single Instruction Multiple Thread (SIMT) mode, or the source operands of instructions in Single Instruction Multiple Data (SIMD) mode.

[0036] The second storage unit stores the source operands of the instructions in the SIMT mode;

[0037] When the execution unit operates in the SIMT mode, it performs calculations based on source operands stored in at least one of the first and second storage units; when it operates in the SIMD mode, it performs calculations based on source operands stored in the first storage unit.

[0038] On the other hand, a computer device is provided, the computer device including a processor and a storage module, the storage module storing at least one computer program, the computer program being loaded and executed by the processor to implement any of the instruction processing methods described above.

[0039] On the other hand, a computer-readable storage medium is provided, wherein at least one computer program is stored therein, the at least one computer program being loaded and executed by a processor to implement any of the instruction processing methods described above.

[0040] On the other hand, a computer program is provided that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the instruction processing methods described above.

[0041] The beneficial effects of the technical solutions provided in this application include at least the following:

[0042] An instruction processing device is designed. Through the collaborative design of the first storage unit and the second storage unit and the unified execution unit, only the first storage unit can be enabled in SIMD mode, and both the first and second storage units can be enabled in SIMT mode. This achieves partial reuse of storage units and full reuse of execution units for the two modes. As a result, it is possible to be compatible with both SIMD and SIMT in one device, while realizing dynamic reuse of hardware resources, reducing deployment costs and improving resource utilization. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a schematic diagram of a computer system provided in an exemplary embodiment of this application;

[0045] Figure 2 This is a schematic diagram of an instruction processing apparatus provided in an exemplary embodiment of this application;

[0046] Figure 3 This is a structural block diagram of an instruction processing apparatus provided in an exemplary embodiment of this application;

[0047] Figure 4 This is a schematic diagram of an instruction processing apparatus provided in another exemplary embodiment of this application;

[0048] Figure 5 This is a schematic diagram of the structure of an instruction processing apparatus provided in an exemplary embodiment of this application;

[0049] Figure 6This is a schematic diagram of the structure of a SIMT and SIMD compatible instruction processing apparatus provided in an exemplary embodiment of this application;

[0050] Figure 7 This is a schematic diagram of an instruction processing apparatus in SIMT mode provided in an exemplary embodiment of this application;

[0051] Figure 8 This is a schematic diagram of an instruction processing apparatus in SIMD mode provided in an exemplary embodiment of this application;

[0052] Figure 9 This is a schematic diagram illustrating resource usage as provided in an exemplary embodiment of this application;

[0053] Figure 10 This is a flowchart of an instruction processing method provided in an exemplary embodiment of this application;

[0054] Figure 11 This is a schematic block diagram of a processor provided in an exemplary embodiment of this application;

[0055] Figure 12 This is a schematic block diagram of a computer device provided in an exemplary embodiment of this application;

[0056] Figure 13 This is a schematic block diagram of a computer device provided in another exemplary embodiment of this application;

[0057] Figure 14 This is a schematic block diagram of a computer device provided in yet another exemplary embodiment of this application;

[0058] Figure 15 This is a schematic block diagram of a computer device provided in another exemplary embodiment of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0060] It should be understood that although the terms first, second, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, a first parameter may also be referred to as a second parameter without departing from the scope of this disclosure, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0061] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user, processor, and computer device data. These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without user confirmation), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of this user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0062] Please refer to Figure 1 , Figure 1 This is a schematic diagram of a computer system provided in an exemplary embodiment of this application. The computer system can be implemented as a hardware architecture of an instruction processing device, or as a system architecture of an instruction processing method applied to the instruction processing device. The computer system includes: a terminal 120 and a server 140.

[0063] In some embodiments, terminal 120 includes the instruction processing device of this embodiment; or, terminal 120 includes a processor, and the processor includes the instruction processing device of this embodiment. Terminal 120 may be an electronic device such as a mobile phone, tablet computer, vehicle terminal (vehicle system), wearable device, personal computer (PC), unmanned reservation terminal, etc., and this application embodiment does not limit it.

[0064] In some embodiments, server 140 includes the instruction processing device of this embodiment; or, server 140 includes a processor, and the processor includes the storage device of this embodiment. Server 140 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud servers, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Server 140 may be a backend server for terminal 120, used to provide backend services for the data operation and processing of terminal 120. Optionally, server 140 may also be implemented as a node in a blockchain system.

[0065] The terminal 120 and the server 140 can communicate via a communication network, such as a wired or wireless network. The instruction processing method applied to an instruction processing device provided in this application embodiment can have its steps executed by an instruction processing device in a computer device, which includes the terminal 120 and / or the server 140.

[0066] Taking the instruction processing device in server 140 as an example, the instruction processing device in server 140 includes: storage unit 10 and execution unit 20. Storage unit 10 is connected to execution unit 20. Storage unit 10 includes a first storage unit 11 and a second storage unit 12.

[0067] Schematic, the storage unit 10 in the instruction processing device of server 140 is used to store the source operands of the instruction; the first storage unit 11 stores the source operands of the instruction in SIMT mode, or stores the source operands of the instruction in SIMD mode; the second storage unit 12 stores the source operands of the instruction in SIMT mode.

[0068] In SIMT mode, the first storage unit 11 and the second storage unit 12 are enabled, while in SIMD mode, only the first storage unit 11 is enabled.

[0069] Schematic, in the instruction processing apparatus of server 140, execution unit 20 performs calculations based on source operands stored in at least one of the first storage unit 11 and the second storage unit 12 when operating in SIMT mode; and in SIMD mode, execution unit 20 performs calculations based on source operands stored in the first storage unit 11.

[0070] AI processors require efficient handling of massively parallel computing tasks. SIMD and SIMT are two mainstream parallel computing architectures, significantly improving computational performance through data-level parallelism and thread-level parallelism, respectively. In related technologies, AI processors typically support only a single architecture (SIMD or SIMT), while operators in AI workloads may be implemented using different architectures due to historical reasons or performance requirements. For example, early operators were developed based on the SIMD architecture, while newly optimized operators may be more suitable for the SIMT architecture. Because of the significant differences in hardware design and software ecosystem between the two architectures, two separate hardware sets are needed to support hybrid operators, such as SIMD accelerators and SIMT GPUs, resulting in high deployment costs and low resource utilization.

[0071] Based on this, this application provides an instruction processing apparatus. Please refer to... Figure 2 , Figure 2 This is a schematic diagram of an instruction processing apparatus provided in an exemplary embodiment of this application.

[0072] like Figure 2 As shown, the instruction processing device includes an address generation unit 210, an instruction buffer unit 220, an instruction dispatch unit 230, an execution unit 240, and a storage unit 250.

[0073] The address generation unit 210 includes a first generation unit 211 and a second generation unit 212. The first generation unit 211 is used to generate program addresses for at least two threads in SIMT mode, and the second generation unit 212 is used to generate program addresses for a single thread in SIMD mode.

[0074] The instruction buffer unit 220 includes a first buffer unit 221 and a second buffer unit 222. The first buffer unit 221 is used to temporarily store instructions to be issued corresponding to at least two threads in the SIMT mode, and the second buffer unit 222 is used to temporarily store instructions to be issued corresponding to the first thread in the SIMD mode.

[0075] The instruction issuing unit 230 is used to obtain instructions from the instruction buffer unit 220 and issue them to the execution unit 240. In the case of operation in SIMT mode, the instruction issuing unit 230 obtains instructions from the first buffer unit 221 and issues them to the execution unit 240; in the case of operation in SIMD mode, the instruction issuing unit 230 obtains instructions from the second buffer unit 222 and issues them to the execution unit 240.

[0076] Storage unit 250 includes a first storage unit 251 and a second storage unit 252. The first storage unit 251 is used to store the source operands of instructions in Single Instruction Multiple Thread (SIMT) mode, or the source operands of instructions in Single Instruction Multiple Data (SIMD) mode. The second storage unit 252 is used to store the source operands of instructions in SIMT mode.

[0077] The execution unit 240 is configured to perform computations based on source operands stored in at least one of the first and second storage units when operating in SIMT mode; and to perform computations based on source operands stored in the first storage unit when operating in SIMD mode.

[0078] In summary, this application proposes a SIMD and SIMT compatible instruction processing device. Through the collaborative design of the first and second storage units and a unified execution unit, it achieves partial reuse of storage units and full reuse of execution units for both modes. This enables the device to be compatible with both SIMD and SIMT while achieving dynamic reuse of hardware resources, reducing deployment costs and improving resource utilization.

[0079] Figure 3This is a structural block diagram of an instruction processing apparatus provided in an exemplary embodiment of this application. The instruction processing apparatus includes a storage unit 310 and an execution unit 320, wherein the storage unit 310 is connected to the execution unit 320, and the storage unit 310 includes a first storage unit 311 and a second storage unit 312.

[0080] The instruction storage device may be implemented as part of a processor, which includes at least one of a GPU and a Central Processing Unit (CPU).

[0081] The processor's processing of a thread (such as the first thread) typically includes two core phases: the resource allocation phase and the instruction execution phase. The resource allocation phase allocates the necessary hardware and software resources to the thread, enabling it to execute. In this application, the resource allocation phase mainly involves the allocation of memory space for the thread. The instruction execution phase executes the thread's instructions, and the execution of these instructions requires the memory space allocated during the resource allocation phase.

[0082] Optionally, the processor described above can be any processor compatible with SIMT and SIMD modes, such as a multi-threaded processor, that is, a processor used for multi-threaded execution scenarios, including GPU parallel matrix multiplication, CPU multi-threaded sorting algorithms, etc., which are not limited here.

[0083] For SIMD mode, applicable scenarios include, but are not limited to, at least one of the following characteristics: high data parallelism, all data elements need to perform the exact same operation; rule-based computation, such as matrix multiplication, vector addition and subtraction, image filtering, etc.; static instruction flow: the instruction sequence is fixed and there is no dynamic thread behavior.

[0084] In SIMD mode, all Arithmetic Logic Units (ALUs) execute synchronously, eliminating thread scheduling overhead and resulting in high hardware efficiency. Furthermore, there is no need to maintain multi-threaded states, leading to lower power consumption. However, SIMD mode struggles to handle branching or conditional statements, exhibiting poor flexibility; it also requires explicit handling of unaligned data, placing high demands on data alignment.

[0085] For the SIMT pattern, applicable scenarios include, but are not limited to, at least one of the following characteristics: thread-level parallelism, where tasks can be decomposed into independent threads, and each thread may follow different branches; irregular computation, such as conditional branches, table lookups, dynamic memory access, etc.; low data dependency, where threads do not need to be frequently synchronized.

[0086] SIMT mode inherently supports branching and thread-independent behavior, offering high flexibility; thread bundle switching can mask memory access latency. However, SIMT mode requires maintaining multi-threaded states, resulting in significant thread management overhead; branching within a thread bundle reduces efficiency and incurs branch penalties.

[0087] In this application embodiment, SIMD mode or SIMT mode can be selected according to the characteristics of the actual application scenario, or a switch can be made between the two.

[0088] To illustrate, for GPU parallel matrix multiplication, regular matrix block operations can be handled in SIMD mode, while irregular sparse computations can be handled in SIMT mode; for CPU multi-threaded sorting algorithms, comparison and swap operations can be performed in parallel in SIMT mode, while key-value processing can be accelerated in SIMD mode.

[0089] It is worth noting that the above-described operation of SIMT and SIMD modes is merely an illustrative example, and the embodiments of this application do not limit this. In the embodiments of this application, a processing mode is pre-configured for a new thread, such as selecting SIMT mode or SIMD mode, and the instruction processing procedure is executed according to the pre-configured mode.

[0090] Storage unit 310 is the Unify regfile of the instruction processing device. It adopts a hierarchical design structure and includes a first storage unit 311 and a second storage unit 312 that are physically independent but logically coordinated.

[0091] The first storage unit 311 is configured to store the source operand of an instruction in Single Instruction Multiple Thread (SIMT) mode, or the source operand of an instruction in Single Instruction Multiple Data (SIMD) mode.

[0092] In some embodiments, the first storage unit 311 is implemented using a high-speed register file, which is a parallel access storage array consisting of at least two registers. It can be used to provide operands required by the arithmetic logic unit (ALU), temporarily store intermediate calculation results, and implement nanosecond-level (typically 1-3 clock cycles) data access.

[0093] The first storage unit 311 is used as a complete register file (regfile) in SIMD mode, storing all vector operands; in SIMT mode, it is used as a register cache, storing only the register data of the active thread bundle.

[0094] In some embodiments, the first storage unit 311 may also support dynamic capacity allocation, which can adjust the effective storage space according to the working mode.

[0095] When a mode switch occurs, the first storage unit 311 can adjust the available storage space. Specifically, this adjustment needs to be made after the pipeline is emptied, i.e., to complete all in-process instructions based on the previous mode. Register remapping is performed: when switching from SIMD to SIMT, the vector register in the first storage unit 311 can be split into thread registers; when switching from SIMT mode to SIMD mode, the active thread registers in the first storage unit 311 can be merged into a vector register.

[0096] Indicatively, the first storage unit 311 may include 256×32 bits of physical registers. In SIMD mode, the first storage unit 311 is used as a complete register file of 64×512 bits through register merging; in SIMT mode, the first storage unit 311 is used as a register cache of 128×256 bits through register merging.

[0097] The second storage unit 312 is configured to store the source operand of instructions in SIMT mode.

[0098] In some embodiments, the second storage unit 312 is implemented using on-chip static random access memory (SRAM).

[0099] SRAM is a type of random access memory whose storage cells are composed of flip-flops made of transistors. As long as power is maintained, the data stored in SRAM can be retained indefinitely, and it has the characteristics of high speed, low power consumption, low integration density, and volatility.

[0100] In some embodiments, SRAM employs a multi-bank architecture to enhance parallel access capabilities; it is equipped with an intelligent prefetch mechanism to reduce data loading latency.

[0101] Taking the first storage unit 311 as an example, which is implemented using a high-speed register, and the second storage unit 312 as an example, the storage density of the second storage unit 312 is significantly higher than that of the first storage unit, but the access latency is higher.

[0102] In this embodiment, the second storage unit 312 is specifically used to store the register state of inactive thread bundles in SIMT mode; in SIMD mode, it can be completely powered off to save power consumption.

[0103] In the SIMD architecture, the register file has fewer registers, and the regfile in the hardware implementation is built using registers. In the SIMT architecture, the register file has more registers, and the regfile in the hardware implementation is built using on-chip SRAM. In the SIMT architecture, in addition to the main regfile, there is a smaller regfile built using registers, called the register cache. This caches recently used data, so if this data is used again later, it does not need to be read from the main regfile, reducing the power consumption of accessing the main regfile.

[0104] In this embodiment of the application, the regfile cache under the SIMT architecture and the regfile under the SIMD architecture are reused, namely the first storage unit 311, which is a small-capacity register file built by registers, and the second storage unit 312 is a large-capacity register file built by SRAM.

[0105] In SIMD mode, only the first storage unit 311 is used as the register file, while in SIMT mode, both the first storage unit 311 and the second storage unit 312 are used. The first storage unit 311 is used as a register cache to cache recently used data, while the second storage unit 312 is used as the main register file.

[0106] The execution unit 320 is configured to perform computations based on source operands stored in at least one of the first and second storage units when operating in SIMT mode; and to perform computations based on source operands stored in the first storage unit when operating in SIMD mode.

[0107] The execution unit 320 is the core processing module in the instruction processing device. In some embodiments, the execution unit includes at least two ALUs, or a Vector Arithmetic Logic Unit (VALU), for performing instruction calculations.

[0108] The execution unit 320 is used to retrieve the source operand from the storage unit 310 according to the instruction, execute the operation corresponding to the instruction to perform the calculation, and store the calculation result in the storage space indicated by the target operand.

[0109] An instruction includes an opcode, a source operand, and a destination operand. The opcode is the part of the instruction that specifies the type of operation, the source operand is the input data that participates in the operation, and the destination operand is the output location where the operation result is stored.

[0110] For example, regarding the instruction "ADD a1, a2, a3", ADD is the opcode used to indicate an addition operation, a1 and a2 are the source operands used to indicate that the data to be calculated is the data stored in registers a1 and a2, and a3 is the destination operand used to indicate that the calculation result is stored in register a3.

[0111] In this embodiment, the execution unit 320 adopts a unified computing architecture design, that is, the hardware structure of the execution unit 320 is the same for both SIMT and SIMD modes.

[0112] In some embodiments, the execution unit 320 may include a large number of dynamically reconfigurable ALUs, all of which share the same instruction decoding and control logic. In SIMD mode, the ALUs are organized as wide-vector processing units, with each ALU executing the same thread instructions in parallel; in SIMT mode, the ALUs are divided into at least two thread processing clusters, which are used to process different thread instructions in parallel.

[0113] In SIMD mode, all ALUs execute a single instruction synchronously to process different data. In SIMT mode, ALUs execute the same instruction in warp bundles.

[0114] Regarding the SIMD mode workflow, when running in SIMD mode, the power supply to the second storage unit 312 is turned off during the system initialization phase; all operand storage and computation are completed in the first storage unit 311; the execution unit 320 configures the computing resources to a wide vector processing mode; the instruction pipeline adopts a deep parallel design; and it supports the synchronous computation of multiple data elements in a single cycle.

[0115] For the SIMT mode workflow, when running in SIMT mode, the second storage unit 312 is powered on and initialized; the thread scheduler manages the execution status of multiple thread bundles; active thread data is preferentially stored in the first storage unit 311; the resources of the execution unit 320 are dynamically allocated according to thread bundles; thread-level parallelism and branch execution are supported.

[0116] In some embodiments, the execution unit 320 is further configured to write the calculation result back to the storage unit 310 after performing a calculation based on the source operand of the instruction.

[0117] That is, the calculation result is written back to the storage unit 310 through the execution unit 320, so that the storage unit 310 can also be used to store intermediate calculation results.

[0118] The execution unit 320 writes the calculation result into the first storage unit 311.

[0119] Specifically, the execution unit 320 is also configured to write the calculation results back to the first storage unit 311 when running in SIMT mode; wherein, when the number of calculation results stored in the first storage unit 311 reaches a preset threshold, the calculation results that were first written to the first storage unit 311 are migrated to the second storage unit 312 according to the first-in-first-out principle.

[0120] The First-In-First-Out (FIFO) principle is used to indicate that the calculation results written back to the first storage unit 311 first will be moved into the second storage unit 312 first.

[0121] To illustrate, when the first storage unit 311 is full, the oldest data is written to the second storage unit 312 to achieve dynamic overflow handling.

[0122] In some embodiments, during the write-back process of the calculation result in SIMT mode, the calculation result can be written back to the storage location in the first storage unit 311 indicated by the target operand in the instruction information, or the calculation result can be written back to an empty slot in the first storage unit 311. This application embodiment does not limit this.

[0123] When the target operand in the instruction information indicates that the storage location is in the second storage unit 312, the calculation result can be written back to the empty slot in the first storage unit 311.

[0124] At this point, in order to ensure that the calculation results can be read accurately, it is necessary to update the mapping table maintained by the first storage unit 311 and the second storage unit 312 in SIMT mode. This mapping table is used to indicate the mapping relationship between the actual storage location in the storage unit 310 in SIMT mode and the storage location indicated in the instruction information.

[0125] In illustratively, in SIMT mode, execution unit 320 obtains the calculation result A for the first instruction. The target operand indicated by the instruction information of the first instruction is b1, which is used to indicate that the calculation result A of the first instruction is stored in register b1. When register b1 is located in the second storage unit 312, the calculation result A can be stored in the empty register b2 in the first storage unit 311. By maintaining the mapping relationship between register b1 and register b2 in the mapping table, the actual target operand of the first instruction is updated to b2. That is, when execution unit 320 executes the first instruction again, it can directly read the calculation result A from register b2 in the first storage unit 311 according to the mapping relationship between the target operand b1 and the mapping table.

[0126] The execution unit 320 is also configured to write the computation results back to the first storage unit 311 when operating in SIMD mode, wherein the second storage unit 312 is disabled.

[0127] Since the second storage unit 312 is disabled in SIMD mode, the calculation result obtained by the execution unit 320 only needs to be written directly to the first storage unit 311. Specifically, the calculation result can be written to the storage location indicated by the target operand in the instruction according to the instruction information.

[0128] After the calculation result is written back, when the same instruction is issued to the execution unit 320 for execution again, the calculation result can be read directly from the storage location indicated by the target operand according to the instruction information until the calculation result is updated.

[0129] A unified write-back strategy ensures data visibility in both modes, simplifies the programming model, and eliminates the need for developers to focus on data storage location, as the hardware automatically manages the write-back process. In SIMT mode, computation results are prioritized for writing back to the first storage unit, and dynamic overflow handling is performed. This allows for prioritizing reading from the first storage unit during data retrieval, increasing the probability of data hits for active instructions and thus improving resource utilization.

[0130] In summary, the embodiments of this application design an instruction processing device. Through the collaborative design of the first storage unit and the second storage unit and a unified execution unit, only the first storage unit can be enabled in SIMD mode, and both the first and second storage units can be enabled in SIMT mode. This achieves partial reuse of storage units and complete reuse of execution units for the two modes, thereby enabling compatibility with SIMD and SIMT in a single device while realizing dynamic reuse of hardware resources, reducing deployment costs, and improving resource utilization.

[0131] Please refer to Figure 4 , Figure 4 This is a schematic diagram of an instruction processing apparatus provided in another exemplary embodiment of this application, such as... Figure 4 As shown, the instruction processing device also includes a reading unit 330, which is connected to the storage unit 310 and the execution unit 320 respectively.

[0132] The read unit 330 is configured to, when operating in SIMT mode, read the source operand of an instruction from at least one of the first storage unit 311 and the second storage unit 312; and send the source operand to the execution unit 320 to perform a computation. When operating in SIMD mode, it reads the source operand of the instruction from the first storage unit 311; and sends the source operand to the execution unit 320 to perform a computation.

[0133] Specifically, the input terminal of the reading unit 330 can be connected to the first storage unit 311 and the second storage unit 312 respectively, and the output terminal of the reading unit 330 is connected to the execution unit 320.

[0134] The read unit is a hardware module in the processor specifically responsible for fetching operands from the memory system. Its core functions include source operand fetching and reading the input data required for instructions from the register file / cache.

[0135] This read unit integrates access logic for both SIMT and SIMD modes, simplifying the data path. It can read the source operands of the instruction from the corresponding memory unit according to different modes and send them to the execution unit to perform calculations, thus simplifying the processing logic of the execution unit.

[0136] In some embodiments, the reading unit 330 needs to determine the reading position of the source operand.

[0137] When the read unit 330 is running in SIMT mode, it reads the source operand of the instruction from the first storage unit 311; if the source operand of the instruction is not included in the first storage unit 311, it reads the source operand of the instruction from the second storage unit 312; when it is running in SIMD mode, it reads the source operand of the instruction from the first storage unit 311, wherein the second storage unit 312 is disabled.

[0138] In SIMT mode, the first storage unit 311 is used to store the register state of the active thread and the second storage unit 312 is used to store the register state of the inactive thread. Therefore, in SIMT mode, when reading the source operand through the read unit, the first storage unit 311 is checked first, and the corresponding register is matched in the first storage unit 311 by the register number. If no match is found, the second storage unit 312 is checked next.

[0139] In SIMD mode, since only the first storage unit 311 is enabled, it is possible to read source operands from the first storage unit 311.

[0140] By prioritizing reading source operands from the first memory unit in SIMT mode, the register data corresponding to the active thread can be checked first, improving the hit rate. If a miss occurs, the source operands are then read from the second memory unit, which improves the data reading efficiency in SIMT mode. By directly disabling the second memory unit in SIMD mode, the static power consumption of the second memory unit can be eliminated, reducing the system power consumption in SIMD mode.

[0141] In some embodiments, the instruction processing apparatus also requires dynamic front-end configuration to ensure compatibility with SIMD and SIMT modes. Please refer to [reference needed]. Figure 5 , Figure 5 This is a schematic diagram of the structure of an instruction processing apparatus provided in an exemplary embodiment of this application, as shown below. Figure 5 As shown, the instruction processing device further includes an address generation unit 340, an instruction buffer unit 350, and an instruction issuing unit 360. The address generation unit 340 is connected to the instruction buffer unit 350, the instruction buffer unit 350 is connected to the instruction issuing unit 360, and the instruction issuing unit 360 is connected to the execution unit 320.

[0142] Address generation unit 340 is configured to generate the program address of a thread. The program address is used to indicate the storage location in memory of the instruction corresponding to the thread.

[0143] A thread is the smallest unit that an operating system can schedule. It consists of a sequence of instructions to be executed, along with the context information required to execute those instructions (such as register states, program counter, etc.). The execution process of a thread can be simply understood as executing instructions sequentially according to the instruction sequence.

[0144] An instruction is the basic unit of operation in a computer program and the smallest unit executed by the CPU. Each instruction tells the CPU to perform a specific operation, such as arithmetic operations, data transfer, or conditional jumps.

[0145] A thread is an organization of the execution sequence of these instructions. A thread can contain one instruction or at least two instructions, wherein the at least two instructions are executed in a certain order.

[0146] A thread can contain at least two instructions, but an instruction typically belongs to only one thread (unless it is a shared instruction among multiple threads, such as an access to a global variable). The sequence of instructions for a thread is determined by the program code, while the execution order and timing of threads are controlled by the operating system's scheduler.

[0147] In some embodiments, the program address is indicated in the form of a program counter (pc), which points to the storage address of the instruction corresponding to the thread.

[0148] Because the thread processing logic in SIMT and SIMD is different, different address generation units need to be designed for SIMT mode and SIMD mode.

[0149] In some embodiments, the address generation unit 340 includes a first generation unit 341 and a second generation unit 342.

[0150] The first generation unit 341 is configured to generate program addresses for at least two threads in SIMT mode.

[0151] The first generation unit 341 is the address generation unit of SIMT, which is used to generate the next program address to be accessed. In SIMT mode, if at least two threads need to be executed in parallel, then the program addresses corresponding to at least two threads need to be obtained respectively.

[0152] In illustrative terms, in SIMT mode, there can be at least two thread bundles, each containing at least two threads. The first generation unit 341 selects one thread bundle pc from the at least two thread bundles each time to fetch instructions (i.e., fetch instructions based on the program address).

[0153] The second generation unit 342 is configured to generate program addresses for the first thread in SIMD mode.

[0154] The second generation unit 342 is the address generation unit of SIMD. In SIMD mode, only one program pointer is maintained, that is, only one thread (such as the first thread) is maintained. The program pointer can be used to directly point to the program address to obtain the instructions corresponding to the thread.

[0155] It is worth noting that during operation, the instruction processing device will be configured to operate in either SIMD mode or SIMT mode. The two modes will not be active at the same time. Therefore, only one of the first generation unit 341 and the second generation unit 342 is in working state at any given time.

[0156] Different address generation units are used to generate program addresses in SIMT and SIMD modes respectively to obtain the instructions corresponding to the threads in the corresponding modes. The first generation unit is used in SIMT mode and the second generation unit is used in SIMD mode, so as to be compatible with different program address generation methods in SIMT and SIMD modes.

[0157] Instruction buffer unit 350 is configured to temporarily store instructions to be issued based on program addresses.

[0158] After obtaining the instruction based on the program address, the instruction needs to be issued to the execution unit 320 for calculation through the instruction issuing unit 360. Since instruction issuance takes time, and at least two instructions corresponding to the thread can be obtained quickly based on the program address, storage space is needed to temporarily store the instructions to be issued. When the instruction issuing unit 360 needs to issue an instruction, it can obtain the instruction from the storage space, i.e., the instruction buffer unit 350.

[0159] In some embodiments, after fetching instructions from memory, storage space is needed to cache the fetched instructions. To simplify the caching logic and avoid affecting the efficiency of memory access, the current mode can be sensed by a multiplexer to determine which address generation unit the program address generated by should be used to fetch the instructions. However, when caching instructions, the current mode does not need to be considered; the fetched instructions and the sense signal of the multiplexer can be cached together.

[0160] In some embodiments, the instruction processing apparatus further includes a multiplexer 370 and an instruction buffer unit 380. The input terminals of the multiplexer 370 are connected to the first generation unit 341 and the second generation unit 342, respectively. The output terminal of the multiplexer 370 is connected to the instruction buffer unit 380, and the instruction buffer unit 380 is connected to the instruction buffer unit 350.

[0161] Multiplexer 370 is configured to send the program address generated by first generation unit 341 to instruction cache unit when operating in SIMT mode; and to send the program address generated by second generation unit 342 to instruction cache unit 380 when operating in SIMD mode.

[0162] Instruction cache unit 380 is configured to fetch instructions based on program addresses; and temporarily store instructions in instruction buffer unit 350.

[0163] The multiplexer is used to select the correct program address source according to the current mode (SIMT or SIMD). In SIMD mode, the output of the second generation unit is selected, and in SIMT mode, the thread bundle pc in the arbiter of the first generation unit is selected.

[0164] The output of the multiplexer is the program address corresponding to the current mode, which is transferred to the instruction cache unit, and the instruction cache unit fetches instructions based on the program address.

[0165] The instruction cache unit is a cache space used to cache instructions, while the instruction buffer unit is a buffer used to temporarily store instructions.

[0166] During the process of temporarily storing instructions in the instruction buffer, the instruction cache unit obtains the program address from the multiplexer, retrieves the instruction based on the program address, and checks whether the program address is in the instruction cache. If a cache hit occurs, the instruction is returned directly; otherwise, the program address is retrieved from the lower-level memory. The obtained instruction is then sent to the instruction buffer unit to await issuance.

[0167] The instruction cache (icache) mentioned above is used to ensure that instructions are always available, and the instruction buffer (ibuffer) is used to ensure that instructions are always available.

[0168] By using a multiplexer to detect different modes, the generation of program addresses and instruction buffering for different modes are handled by the multiplexer and instruction buffer unit respectively, simplifying instruction caching logic and improving memory access efficiency.

[0169] In some embodiments, since SIMD and SIMT use different parallel processing methods, buffers with different structures are required to temporarily store cached instructions.

[0170] In some embodiments, the instruction buffer unit 350 includes a first buffer unit 351 and a second buffer unit 352.

[0171] The first buffer unit 351 includes at least two buffer queues, and the i-th buffer queue in the at least two buffer queues is used to temporarily store the instructions corresponding to the i-th thread in at least two threads in the SIMT mode, where i is a positive integer; the second buffer unit 352 includes a single queue, and the single queue is used to temporarily store the instructions corresponding to the first thread in the SIMD mode.

[0172] Instruction cache unit 380 is also configured to temporarily store instructions in first buffer unit 351 when operating in SIMT mode.

[0173] In SIMT mode, instruction cache unit 380 temporarily stores instructions in a first buffer unit 351, which includes at least two buffer queues, so that at least two threads can execute the same instructions in parallel, while each thread can process different data independently.

[0174] In SIMT mode, threads are organized into thread groups (such as Warp), and each Warp contains multiple threads (e.g., 32 threads), which can execute independently.

[0175] At least two buffer queues are used, each temporarily storing instructions for the corresponding thread, ensuring that instructions can be distributed to each thread in order, and supporting thread-level parallel execution.

[0176] SIMT mode allows each thread to execute independently and supports branch divergence, meaning different threads can execute different branches.

[0177] By employing the first buffer unit described above in SIMT mode, thread-level instruction dispatch is supported through at least two buffer queues, with each queue corresponding to one thread, ensuring that threads can execute independently. This fully adapts to multi-threaded concurrent processing in SIMT mode and improves the instruction processing efficiency of the instruction processing device in SIMT mode.

[0178] Instruction cache unit 380 is also configured to temporarily store instructions in second buffer unit 352 when operating in SIMD mode.

[0179] In SIMD mode, instruction cache unit 380 temporarily stores instructions in a second buffer unit 352, which includes a single queue, in line with the characteristic of SIMD mode that focuses on executing the same instructions in parallel on multiple data elements.

[0180] By employing the second buffer unit described above in SIMD mode, parallel operations on multiple data elements can be supported through a single queue temporary storage instruction.

[0181] By employing different instruction buffer structures in different modes, it is possible to adapt to the needs of different modes. At least two buffer queues can support thread-level instruction dispatch in SIMT mode, while a single queue can support multi-data parallel operations in SIMD mode, thereby improving the compatibility of the instruction processing device with SIMT and SIMD in terms of instruction buffering.

[0182] The instruction issuing unit 360 is configured to issue instructions temporarily stored in the instruction buffer unit to the execution unit for computation.

[0183] In some embodiments, the instruction issuing unit 360 is configured to issue instructions temporarily stored in the first buffer unit to the execution unit for computation in SIMT mode, and to issue instructions temporarily stored in the second buffer unit to the execution unit for computation in SIMD mode.

[0184] Since the instruction processing device is only in one of the SIMT or SIMD modes at any given time, it needs to wait for the corresponding instruction in one of the modes to be processed before it can switch modes and enter a new instruction processing flow. Therefore, in the instruction buffer unit, only one of the first buffer unit and the second buffer unit is in working state at any given time. That is, at most only one buffer unit can temporarily store the instruction to be transmitted at any given time.

[0185] When issuing commands, the command issuing unit only needs to retrieve the temporarily stored commands from the command buffer that is in the working state and issue them. There is no need to determine the current mode type of the device. Therefore, the command buffer unit based on the hierarchical structure (including the first buffer unit in SIMT mode and the second buffer unit in SIMD mode) can simplify the command issuing logic and improve the command issuing efficiency while achieving SIMT and SIMD compatibility.

[0186] In summary, the apparatus provided in this application provides an apparatus structure including an address generation unit, an instruction cache unit, and an instruction dispatch unit, realizing a three-stage pipeline design for address generation, caching, and dispatching. This ensures efficient supply of instruction streams, thereby improving the parallel processing efficiency of the instruction processing device, reducing resource idleness, and increasing resource utilization.

[0187] The following explanation uses an instruction processing device that is compatible with both SIMT and SIMD based on front-end dynamic switching and back-end hardware reuse as an example. Figure 6 This is a schematic diagram of the structure of a SIMT and SIMD compatible instruction processing apparatus provided in an exemplary embodiment of this application.

[0188] like Figure 6 As shown, the instruction processing device includes: a storage unit 310, an execution unit 320, a read unit 330, an address generation unit 340, an instruction buffer unit 350, an instruction dispatch unit 360, a multiplexer 370, and an instruction cache unit 380.

[0189] The storage unit 310 includes a first storage unit 311 and a second storage unit 312, the address generation unit 340 includes a first generation unit 341 and a second generation unit 342, and the instruction buffer unit 350 includes a first buffer unit 351 and a second buffer unit 352.

[0190] • Dynamic switching of the front end.

[0191] In dynamic switching on the front end, it is necessary to obtain the corresponding instructions according to different modes.

[0192] Specifically, this includes compatibility improvements in address generation, instruction buffering, and instruction issuance.

[0193] In SIMT mode, the first generation unit 341 (simt_pc_gen) manages at least two thread bundles of PCs, and the active thread bundle is selected by the arbiter.

[0194] An independent queue is maintained for each thread bundle through instruction buffer unit 380 (ibuffer).

[0195] In SIMD mode, a single pc is generated using the second generation unit 341.

[0196] The instruction cache (icache) and instruction buffer (ibuffer) maintain only a single-threaded stream.

[0197] `simt_pc_gen` is the program address generator (PC) in SIMT mode, used to generate the next program address to be accessed. The SIMT architecture has multiple thread bundles, and each time the PC of one of these bundles is selected for instruction fetching. `pc0` to `pct-1` represent the program pointers of `t` thread bundles, and `arbiter` is the arbiter, which can select one of the `t` thread bundles for execution based on different arbitration logic. `simd_pc_gen` is the program address generator in the SIMD architecture. The SIMD architecture maintains only one program pointer, using a Branch Prediction Unit (BPU) to predict the next program pointer.

[0198] Mux selects the outputs of simt_pc_gen and simd_pc_gen. During the execution of a certain operator, only one of the two program address generation units is working. That is, during program execution, the entire AI processor will be configured in either SIMT mode or SIMD mode. The two modes will not be active at the same time.

[0199] Mux sends the program address to the icache, which is an instruction cache. Based on the program address, the corresponding instruction is retrieved from the icache. If the icache does not find the address, it will continue to search for the program address in the lower-level cache until the address is found.

[0200] After the cache retrieves the instruction, it sends the instruction to the ibuffer. There are two ibuffers: one is simt_ibuffer. In the SIMT architecture, there are multiple thread bundles, so each thread bundle is equipped with a queue. In the SIMD architecture, there is only one thread, so there is only one ibuffer.

[0201] The Issue unit is the issuing unit. After obtaining instructions from the ibuffer, it issues the instructions to the ALU.

[0202] • Reuse of backend hardware.

[0203] Back-end hardware reuse includes register file reuse and computing unit reuse.

[0204] For register file reuse, a Unify regfile is used, i.e., storage unit 310. In the SIMD architecture, the register file has fewer registers, and the regfile is built using registers in the hardware implementation. In the SIMT architecture, the register file has many registers, and the regfile is built using on-chip SRAM in the hardware implementation. In the SIMT architecture, in addition to the main regfile, there is a smaller regfile built using registers, called the regfile cache, used to cache recently used data. If this data is used again later, it does not need to be read from the main regfile, reducing the power consumption of accessing the main regfile. Here, the regfile cache in the SIMT architecture and the regfile in the SIMD architecture are reused, corresponding to rf_a, i.e., the first storage unit 311. rf_a is the smaller register file built using registers, and rf_b is the larger register file built using SRAM, i.e., the second storage unit 312. In SIMD mode, only rf_a is used as the register file, while in SIMT mode, both rf_a and rf_b are used. rf_a serves as the regfile cache, caching recently used data, while rf_b serves as the main regfile.

[0205] The Op read (operand read) unit is the operand read unit, which reads register data from rf_a / rf_b.

[0206] The read operands are sent to valu for arithmetic and logical operations. Each VALU contains at least two ALUs, and at least two ALUs execute the same instruction. After the data processing is completed, the calculation result is written back to regfile.

[0207] Next, based on the above-mentioned instruction processing device compatible with SIMT and SIMD, the processing flow of the instruction processing device in SIMT and SIMD modes will be explained respectively.

[0208] I. Instruction processing flow in SIMT mode.

[0209] Please refer to Figure 7 , Figure 7 This is a schematic diagram of an instruction processing apparatus in SIMT mode provided in an exemplary embodiment of this application, as shown below. Figure 7 As shown, during instruction processing in SIMT mode, the second generation unit 342 and the second buffer unit 352 are disabled.

[0210] In particular, the disabled components in the instruction processing unit can be powered off to reduce energy consumption.

[0211] The instruction processing procedure in SIMT mode includes the following steps:

[0212] (1) Select one of the at least two pcs in the first generation unit 341 to perform instruction fetch operation, wherein the pc of the active thread bundle is selected by the arbiter and input to the multiplexer 370 (mux), and the multiplexer 370 inputs the program address to the instruction cache unit 380 (icache).

[0213] (2) Obtain the instruction from the instruction cache unit 380 (icache) and send the instruction to the corresponding instruction buffer unit 350 in the buffer unit corresponding to the SIMT mode, that is, send it to the first buffer unit 351.

[0214] (3) The instruction issuing unit 360 selects an instruction from the first buffer unit 351 and issues it.

[0215] (4) The execution unit 320 calls the reading unit 330 according to the instruction information to read the source operand from the first storage unit 311 (rf_a) or the second storage unit 312 (rf_b), and the reading unit 330 sends the read operand to the execution unit 320 for execution.

[0216] (5) The execution unit 320 writes the calculation result back to the first storage unit 311 (rf_a);

[0217] (6) When the data in the first storage unit 311 (rf_a) is full, the oldest data is written to the second storage unit 312 (rf_b) to realize dynamic overflow processing.

[0218] II. Instruction processing flow in SIMD mode.

[0219] Please refer to Figure 8 , Figure 8 This is a schematic diagram of an instruction processing apparatus in SIMD mode provided in an exemplary embodiment of this application, as shown below. Figure 8 As shown, during instruction processing in SIMD mode, the first generation unit 341, the first buffer unit 351, and the second storage unit 312 are disabled.

[0220] In particular, the disabled components in the instruction processing unit can be powered off to reduce energy consumption.

[0221] The instruction processing procedure in SIMD mode includes the following steps:

[0222] (1) Calculate the address of the next instruction based on the branch prediction result in the second generation unit 342, and perform instruction fetch operation based on the pc. The pc obtained from the branch prediction result is input to the multiplexer 370 (mux), and the multiplexer 370 inputs the program address to the instruction cache unit 380 (icache).

[0223] (2) Obtain the instruction from the instruction cache unit 380 (icache) and send the instruction to the corresponding instruction buffer unit 350, which is the buffer unit corresponding to the SIMD mode, that is, send it to the second buffer unit 352.

[0224] (3) The command issuing unit 360 selects the command in the head from the second buffer unit 352 and issues it.

[0225] (4) The execution unit 320 calls the reading unit 330 according to the instruction information to read the source operand from the first storage unit 311 (rf_a), and the reading unit 330 sends the read operand to the execution unit 320 for operation.

[0226] (5) The execution unit 320 performs calculations based on the source operands to obtain the calculation results.

[0227] (6) The execution unit 320 writes the calculation result back to the first storage unit 311 (rf_a).

[0228] Based on the above-mentioned instruction processing device compatible with both SIMT and SIMD modes, the resource usage of this device will be explained.

[0229] Please refer to Figure 9 , Figure 9 This is a schematic diagram illustrating resource usage as provided in an exemplary embodiment of this application, such as... Figure 9 As shown in resource usage 900, the resource usage ratio is expressed by area. Among them, the address generation unit 901 (pc_gen), instruction cache unit 902 (icache), instruction buffer unit 903 (ibuffer), and instruction issue unit 904 (issue) in the front end have relatively small resource usage, while the modules with larger resource usage are mainly the storage unit 905 (regfile) and the execution unit 906 (valu).

[0230] In the instruction processing apparatus provided in this application embodiment, all resources of valu are reused for both SIMT and SIMD modes. In regfile, the regfile cache in SIMT mode and the regfile in SIMD mode are reused.

[0231] It is clear that a large amount of resources have been reused in regfile and valu, which account for a large proportion of the area, so that only a small amount of front-end logic needs to be added to complete the support for SIMT mode and SIMD mode.

[0232] In summary, the apparatus provided in this application proposes an instruction processing device design that is compatible with both SIMD and SIMT architectures. It can support both SIMD and SIMT modes simultaneously with minimal hardware resource consumption, thereby significantly reducing hardware deployment costs.

[0233] This is illustrative; please refer to it. Figure 10 This document illustrates a flowchart of an instruction processing method provided in an exemplary embodiment of this application. This method can be executed by a terminal, a server, or both simultaneously. This embodiment describes the method as being executed by an instruction processing device. The instruction processing device includes a storage unit and an execution unit; the storage unit is connected to the execution unit; the storage unit includes a first storage unit and a second storage unit, such as... Figure 10 As shown, the method includes the following steps:

[0234] Step 1010: The first storage unit stores the source operand of the instruction in Single Instruction Multiple Thread (SIMT) mode, or the source operand of the instruction in Single Instruction Multiple Data (SIMD) mode.

[0235] The first storage unit selects one storage location based on the current mode of the instruction processing device.

[0236] In SIMT mode, the first storage unit is used as a register cache, storing only the register data of the active thread bundle. In SIMD mode, the first storage unit is used as a complete register file.

[0237] Step 1020: The second storage unit stores the source operand of the instruction in SIMT mode.

[0238] The second storage unit is a dedicated storage unit for SIMT mode, which is enabled only in SIMT mode, i.e., disabled in SIMD mode.

[0239] In some embodiments, the second storage unit can be disabled by powering down in SIMD mode, thereby eliminating the power consumption of the second storage unit and avoiding unnecessary energy waste.

[0240] In the SIMD architecture, the register file has fewer registers, and the regfile in the hardware implementation is built using registers. In the SIMT architecture, the register file has more registers, and the regfile in the hardware implementation is built using on-chip SRAM. In the SIMT architecture, in addition to the main regfile, there is a smaller regfile built using registers, called the register cache. This caches recently used data, so if this data is used again later, it does not need to be read from the main regfile, reducing the power consumption of accessing the main regfile.

[0241] In this embodiment, the regfile cache under the SIMT architecture and the regfile under the SIMD architecture are reused, namely the first storage unit. The first storage unit is a small-capacity register file built from registers, and the second storage unit is a large-capacity register file built from SRAM.

[0242] In SIMD mode, only the first storage unit is used as the register file, while in SIMT mode, both the first and second storage units are used. The first storage unit serves as a register cache, caching recently used data, while the second storage unit serves as the main register file.

[0243] For other implementation details, please refer to the above embodiments, which will not be repeated here.

[0244] Step 1030: When the execution unit is running in SIMT mode, it performs calculations based on source operands stored in at least one of the first and second storage units; when it is running in SIMD mode, it performs calculations based on source operands stored in the first storage unit.

[0245] In some embodiments, the storage space of the first storage unit is smaller than the storage space of the second storage unit, and the storage density of the second storage unit is higher than that of the first storage unit.

[0246] In the case of running in SIMT mode, when performing calculations based on source operands stored in at least one of the first and second storage units, calculations are performed preferentially based on source operands stored in the first storage unit. That is, the source operands of the instruction are read from the first storage unit first, and if the instruction is not found in the first storage unit, the source operands of the instruction are read from the second storage unit.

[0247] When operating in SIMD mode, computation is performed based on the source operands stored in the first memory unit, that is, the source operands of the instruction are read directly from the first memory unit, wherein the second memory unit is disabled.

[0248] In some embodiments, the device further includes a reading unit connected to both the storage unit and the execution unit.

[0249] Prior to step 1030, the method further includes: when the read unit is operating in SIMT mode, reading the source operand of the instruction from at least one of the first and second storage units; sending the source operand to the execution unit to perform computation; and when the read unit is operating in SIMD mode, reading the source operand of the instruction from the first storage unit; sending the source operand to the execution unit to perform computation.

[0250] In some embodiments, the source operand reading process in SIMT mode includes: reading the source operand of the instruction from a first storage unit when operating in SIMT mode; and reading the source operand of the instruction from a second storage unit when the first storage unit does not contain the source operand of the instruction.

[0251] The source operand reading process in SIMD mode includes: when the read unit is running in SIMD mode, it reads the source operand of the instruction from the first memory unit, wherein the second memory unit is disabled.

[0252] In some embodiments, after step 1030, a calculation result write-back process is further included, which includes: after the execution unit performs calculation based on the source operand of the instruction, the calculation result is written back to the storage unit.

[0253] Specifically, when the execution unit is running in SIMT mode, it writes the calculation results back to the first storage unit; wherein, when the number of calculation results stored in the first storage unit reaches a preset threshold, the calculation results that were first written to the first storage unit are migrated to the second storage unit according to the first-in-first-out principle.

[0254] When the execution unit is running in SIMD mode, it writes the computation results back to the first storage unit, where the second storage unit is disabled.

[0255] In some embodiments, the apparatus further includes an address generation unit, an instruction cache unit, and an instruction issuing unit, wherein the address generation unit is connected to the instruction cache unit, the instruction cache unit is connected to the instruction issuing unit, and the instruction issuing unit is connected to the execution unit.

[0256] Before step 1030, the program also includes an address generation unit generating a program address for the thread, which indicates the storage location of the instruction corresponding to the thread in memory; an instruction buffer unit temporarily storing the instruction to be issued based on the program address; and an instruction issuing unit issuing the instruction temporarily stored in the instruction buffer unit to the execution unit to perform the calculation.

[0257] In some embodiments, the address generation unit includes a first generation unit and a second generation unit; the address generation process includes: the first generation unit generating program addresses for at least two threads in SIMT mode respectively; and the second generation unit generating program addresses for the first thread in SIMD mode.

[0258] In some embodiments, the device further includes a multiplexer and an instruction cache unit. The input of the multiplexer is connected to the first generation unit and the second generation unit, respectively, and the output of the multiplexer is connected to the instruction cache unit. The instruction cache unit is connected to the instruction buffer unit.

[0259] After generating the program address, the process also includes: when the multiplexer is running in SIMT mode, sending the program address generated by the first generation unit to the instruction cache unit; when it is running in SIMD mode, sending the program address generated by the second generation unit to the instruction cache unit; the instruction cache unit fetching instructions based on the program address; and temporarily storing the instructions in the instruction buffer unit.

[0260] In some embodiments, the instruction buffer unit includes a first buffer unit and a second buffer unit. The first buffer unit includes at least two buffer queues, and the i-th buffer queue in the at least two buffer queues is used to temporarily store the instructions corresponding to the i-th thread in at least two threads in the SIMT mode, where i is a positive integer. The second buffer unit includes a single queue, and the single queue is used to temporarily store the instructions corresponding to the first thread in the SIMD mode.

[0261] Storing instructions temporarily in the instruction buffer unit includes: when the instruction buffer unit is operating in SIMT mode, storing instructions temporarily in the first buffer unit; when the instruction buffer unit is operating in SIMD mode, storing instructions temporarily in the second buffer unit.

[0262] For specific implementation details, please refer to the above embodiments, which will not be repeated here.

[0263] In summary, the method provided in this application, through the collaborative design of the first storage unit and the second storage unit in the instruction processing device and a unified execution unit, can enable only the first storage unit in SIMD mode and enable both the first and second storage units in SIMT mode. This achieves partial reuse of storage units and complete reuse of execution units for the two modes, thereby enabling compatibility with both SIMD and SIMT in a single device while realizing dynamic reuse of hardware resources, reducing deployment costs, and improving resource utilization.

[0264] This application provides a processor. Figure 11This is a schematic block diagram of a processor provided in an exemplary embodiment of this application. The processor 1100 includes the instruction processing apparatus provided in the above embodiments. Alternatively, an embodiment of this application provides a processor including programmable logic circuits and / or program instructions, which, when running on a computer device, are used to implement the instruction processing method provided in the above method embodiments.

[0265] Processor 1100 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1100 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 1100 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1100 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1100 may also include an Artificial Intelligence (AI) processor, which is used to handle computational operations related to machine learning.

[0266] This application provides a computer device, which can be implemented as a terminal or a server. Figure 12 This is a schematic block diagram of a computer device provided in an exemplary embodiment of this application. The computer device 1200 includes the instruction processing apparatus provided in the above embodiment. Alternatively, Figure 13 This is a schematic block diagram of a computer device provided in an exemplary embodiment of this application. The computer device 1300 includes the processor provided in the above embodiments. Optionally, this application also provides a server cluster, which includes at least two computer devices, including: the instruction processing apparatus provided in the above embodiments, or, a processor as provided in the above embodiments.

[0267] This application provides a computer device, comprising: a processor and a storage module, wherein the storage module stores a computer program; and a processor for executing the computer program in the storage module to implement the instruction processing methods provided in the above-described method embodiments. Optionally, the instruction processing device described above is a device on the processor, and the storage module is a module outside the processor.

[0268] This application provides a computer-readable storage medium storing a computer program, which is loaded and executed by a processor to implement the instruction processing methods provided in the above-described method embodiments.

[0269] This application provides a computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the instruction processing methods provided in the above-described method embodiments.

[0270] Figure 14 This is a schematic block diagram of a computer device provided in an exemplary embodiment of this application. The computer device is a server 1400. Typically, the server 1400 includes a processor 1401 and a storage module 1402.

[0271] Storage module 1402 may include one or more computer-readable storage media, which may be non-transitory. Storage module 1402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in storage module 1402 is used to store at least one computer program, which is executed by processor 1401 to implement the instruction processing method provided in the method embodiments of this application.

[0272] In some embodiments, the server 1400 may optionally include an input interface 1403 and an output interface 1404. The processor 1401, storage module 1402, and input interface 1403 and output interface 1404 can be connected via a bus or signal lines. Various peripheral devices can be connected to the input interface 1403 and output interface 1404 via a bus, signal lines, or a circuit board. The input interface 1403 and output interface 1404 can be used to connect at least one input / output (I / O) related peripheral device to the processor 1401 and storage module 1402. In some embodiments, the processor 1401, storage module 1402, and input interface 1403 and output interface 1404 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1401, storage module 1402, and input interface 1403 and output interface 1404 can be implemented on separate chips or circuit boards, and this application embodiment does not limit this.

[0273] Figure 15 This is a schematic block diagram of a computer device provided in an exemplary embodiment of this application. Optionally, the computer device 1500 is a terminal.

[0274] The computer device 1500 can be a portable mobile terminal, also referred to as a mobile terminal in this embodiment. Examples include smartphones, tablets, MP3 players (Moving Picture Experts Group Audio Layer III), and MP4 players (Moving Picture Experts Group Audio Layer IV). The computer device 1500 may also be referred to as user equipment, portable terminal, or other names.

[0275] Typically, computer device 1500 includes a processor 1501 and a storage module 1502.

[0276] Storage module 1502 may include one or more computer-readable storage media, which may be non-transitory. Storage module 1502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in storage module 1502 is used to store at least one computer program, which is executed by processor 1501 to implement the instruction processing method provided in the method embodiments of this application.

[0277] In some embodiments, the computer device 1500 may also optionally include: a peripheral device interface 1503 and at least one peripheral device. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1504, a touch display screen 1505, a camera assembly 1506, an audio circuit 1507, and a power supply 1508. The computer device 1500 also includes one or more sensors 1509. The one or more sensors 1509 include, but are not limited to: an accelerometer 1510, a gyroscope 1511, a pressure sensor 1512, an optical sensor 1513, and a proximity sensor 1514.

[0278] Those skilled in the art will understand that Figure 11 The structure shown does not constitute a limitation on the processor. Figure 12 , Figure 13 , Figure 14 and Figure 15 The structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0279] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0280] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0281] Those skilled in the art will recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of a computer program from one place to another. Storage media can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0282] It should be noted that the collection and processing of relevant data (such as proprietary data, shared data, etc.) in this application should strictly comply with the requirements of relevant national laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0283] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An instruction processing device, characterized in that, The instruction processing device includes a storage unit and an execution unit; the storage unit is connected to the execution unit; the storage unit includes a first storage unit and a second storage unit. The first storage unit is configured to store the source operands of instructions in Single Instruction Multiple Thread (SIMT) mode, or to store the source operands of instructions in Single Instruction Multiple Data (SIMD) mode. The second storage unit is configured to store the source operands of the instructions in the SIMT mode; The execution unit is configured to perform computations based on source operands stored in at least one of the first and second storage units when operating in the SIMT mode; and to perform computations based on source operands stored in the first storage unit when operating in the SIMD mode.

2. The apparatus according to claim 1, characterized in that, The device further includes a reading unit, which is connected to both the storage unit and the execution unit. The read unit is configured to read the source operand of the instruction from at least one of the first and second memory units when operating in the SIMT mode; The source operands are sent to the execution unit to perform the calculation; The read unit is also configured to read the source operand of the instruction from the first storage unit when operating in the SIMD mode; The source operands are sent to the execution unit to perform the calculation.

3. The apparatus according to claim 2, characterized in that, The read unit is further configured to read the source operand of the instruction from the first storage unit when operating in the SIMT mode; and to read the source operand of the instruction from the second storage unit when the first storage unit does not contain the source operand of the instruction. The read unit is further configured to read the source operand of the instruction from the first storage unit when operating in the SIMD mode, wherein the second storage unit is disabled.

4. The apparatus according to any one of claims 1 to 3, characterized in that, The execution unit is further configured to write the calculation result back to the storage unit after performing a calculation based on the source operand of the instruction.

5. The apparatus according to claim 4, characterized in that, The execution unit is further configured to write the calculation result back to the first storage unit when running in the SIMT mode; wherein, when the number of calculation results stored in the first storage unit reaches a preset threshold, the calculation result earliest written to the first storage unit is migrated to the second storage unit according to the first-in-first-out principle. The execution unit is further configured to write the computation result back to the first storage unit when operating in the SIMD mode, wherein the second storage unit is disabled.

6. The apparatus according to any one of claims 1 to 3, characterized in that, The device further includes an address generation unit, an instruction buffer unit, and an instruction issuing unit. The address generation unit is connected to the instruction buffer unit, the instruction buffer unit is connected to the instruction issuing unit, and the instruction issuing unit is connected to the execution unit. The address generation unit is configured to generate the program address of a thread, and the program address is used to indicate the storage location in memory of the instruction corresponding to the thread. The instruction buffer unit is configured to temporarily store instructions to be issued based on the program address; The instruction issuing unit is configured to issue instructions temporarily stored in the instruction buffer unit to the execution unit to perform calculations.

7. The apparatus according to claim 6, characterized in that, The address generation unit includes a first generation unit and a second generation unit; The first generation unit is configured to generate the program address for at least two threads in the SIMT mode respectively; The second generation unit is configured to generate the program address for the first thread in the SIMD mode.

8. The apparatus according to claim 7, characterized in that, The device further includes a multiplexer and an instruction cache unit. The input of the multiplexer is connected to the first generation unit and the second generation unit, respectively. The output of the multiplexer is connected to the instruction cache unit, and the instruction cache unit is connected to the instruction buffer unit. The multiplexer is configured to send the program address generated by the first generation unit to the instruction cache unit when operating in the SIMT mode; and to send the program address generated by the second generation unit to the instruction cache unit when operating in the SIMD mode. The instruction cache unit is configured to retrieve the instruction based on the program address and temporarily store the instruction in the instruction buffer unit.

9. The apparatus according to claim 8, characterized in that, The instruction buffer unit includes a first buffer unit and a second buffer unit. The first buffer unit includes at least two buffer queues. The i-th buffer queue in the at least two buffer queues is used to temporarily store the instructions corresponding to the i-th thread in the at least two threads in the SIMT mode, where i is a positive integer. The second buffer unit includes a single queue, which is used to temporarily store the instructions corresponding to the first thread in the SIMD mode. The instruction cache unit is further configured to temporarily store the instructions in the first buffer unit when operating in the SIMT mode; The instruction cache unit is further configured to temporarily store the instructions in the second buffer unit when operating in the SIMD mode.

10. A processor, characterized in that, The processor includes: an instruction processing apparatus as described in any one of claims 1 to 9.

11. A computer device, characterized in that, The computer device includes: an instruction processing apparatus as described in any one of claims 1 to 9, or a processor as described in claim 10.

12. A server cluster, characterized in that, The server cluster includes at least two computer devices, each including an instruction processing apparatus as described in any one of claims 1 to 9, or a processor as described in claim 10.

13. An instruction processing method, characterized in that, The method is applied to an instruction processing device, which includes a storage unit and an execution unit; the storage unit is connected to the execution unit; the storage unit includes a first storage unit and a second storage unit. The method includes: The first storage unit stores the source operands of instructions in Single Instruction Multiple Thread (SIMT) mode, or the source operands of instructions in Single Instruction Multiple Data (SIMD) mode. The second storage unit stores the source operands of the instructions in the SIMT mode; When the execution unit operates in the SIMT mode, it performs calculations based on source operands stored in at least one of the first and second storage units; when it operates in the SIMD mode, it performs calculations based on source operands stored in the first storage unit.

14. The method according to claim 13, characterized in that, The device further includes a reading unit, which is connected to both the storage unit and the execution unit. When the execution unit operates in the SIMT mode, it performs computation based on source operands stored in at least one of the first and second storage units; when operating in the SIMD mode, before performing computation based on the source operands stored in the first storage unit, it further includes: When the read unit operates in the SIMT mode, it reads the source operand of the instruction from at least one of the first and second storage units; and sends the source operand to the execution unit to perform the calculation. When the reading unit operates in the SIMD mode, it reads the source operand of the instruction from the first storage unit and sends the source operand to the execution unit to perform the calculation.

15. The method according to claim 14, characterized in that, When the read unit is operating in the SIMT mode, it reads the source operand of the instruction from at least one of the first and second memory units. Sending the source operands to the execution unit to perform the computation includes: The read unit reads the source operand of the instruction from the first storage unit when operating in the SIMT mode; and reads the source operand of the instruction from the second storage unit when the first storage unit does not contain the source operand of the instruction. When the reading unit operates in the SIMD mode, it reads the source operand of the instruction from the first storage unit; and sends the source operand to the execution unit to perform the calculation, including: When the read unit is operating in the SIMD mode, it reads the source operand of the instruction from the first storage unit, wherein the second storage unit is disabled.

16. The method according to any one of claims 13 to 15, characterized in that, The method further includes: When the execution unit is running in the SIMT mode, it writes the calculation result back to the first storage unit; wherein, when the number of calculation results stored in the first storage unit reaches a preset threshold, the calculation result earliest written to the first storage unit is migrated to the second storage unit according to the first-in-first-out principle. When the execution unit is running in the SIMD mode, it writes the calculation result back to the first storage unit, wherein the second storage unit is disabled.

17. The method according to any one of claims 13 to 15, characterized in that, The device further includes an address generation unit, an instruction buffer unit, and an instruction issuing unit. The address generation unit is connected to the instruction buffer unit, the instruction buffer unit is connected to the instruction issuing unit, and the instruction issuing unit is connected to the execution unit. The method further includes: The address generation unit generates the program address of the thread, and the program address is used to indicate the storage location of the instruction corresponding to the thread in memory; The instruction buffer unit temporarily stores the instructions to be issued, obtained based on the program address; The instruction issuing unit sends the instructions temporarily stored in the instruction buffer unit to the execution unit to perform the calculation.

18. The method according to claim 17, characterized in that, The address generation unit includes a first generation unit and a second generation unit; The address generation unit generates the program address of the thread, including: The first generation unit generates the program address for at least two threads in the SIMT mode respectively; The second generation unit generates the program address for the first thread in the SIMD mode.

19. The method according to claim 18, characterized in that, The device further includes a multiplexer and an instruction cache unit. The input of the multiplexer is connected to the first generation unit and the second generation unit, respectively. The output of the multiplexer is connected to the instruction cache unit, and the instruction cache unit is connected to the instruction buffer unit. After the address generation unit generates the program address of the thread, it also includes: When the multiplexer is operating in the SIMT mode, it sends the program address generated by the first generation unit to the instruction cache unit. When operating in the SIMD mode, the program address generated by the second generation unit is sent to the instruction cache unit; The instruction cache unit obtains the instruction based on the program address and temporarily stores the instruction in the instruction buffer unit.

20. The method according to claim 19, characterized in that, The instruction buffer unit includes a first buffer unit and a second buffer unit. The first buffer unit includes at least two buffer queues. The i-th buffer queue in the at least two buffer queues is used to temporarily store the instructions corresponding to the i-th thread in the at least two threads in the SIMT mode, where i is a positive integer. The second buffer unit includes a single queue, which is used to temporarily store the instructions corresponding to the first thread in the SIMD mode. The step of temporarily storing the instruction in the instruction buffer unit includes: When the instruction cache unit is operating in the SIMT mode, it temporarily stores the instruction in the first buffer unit; When the instruction cache unit is operating in the SIMD mode, it temporarily stores the instructions in the second buffer unit.

Images