Instruction processing apparatus and method, and processor, chip and board

Abstract

An instruction processing apparatus and method, and a processor, a chip and a board. The instruction processing apparatus comprises: a decoding unit, which is configured to decode an instruction; a micro-instruction generation unit, which is configured to decompose the decoded instruction into several micro-instructions; a micro-instruction-level dependency relationship determination unit, which is configured to determine whether there is a micro-instruction-level dependency relationship between a currently processed micro-instruction and a preceding instruction in execution; and an instruction issue unit, which is configured to determine, on the basis of the micro-instruction-level dependency relationship, whether to issue the currently processed micro-instruction. The instruction processing apparatus provides micro-instruction-level dependency relationship determination, enables more fine-grained scheduling of instructions for parallel execution, and reduces the latency between dependent instructions, thereby improving the instruction parallelism and the processor performance.

Description

Instruction processing device, method, processor, chip and board

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese patent application filed on December 30, 2024, with application number 202411998948.0 and entitled "Instruction Processing Apparatus, Method, Processor, Chip and Board". Technical Field

[0003] This disclosure generally relates to the field of processors. More specifically, this disclosure relates to an instruction processing apparatus, an instruction processing method, a processor, a chip, and a circuit board. Background Technology

[0004] With the development of computer technology, the computing power of processors has continued to improve. The design of single-core processors, which improves performance by increasing frequency, has encountered a bottleneck due to power consumption limitations. Multi-core processors have gradually taken over the market. In multi-core processors, tasks can be distributed to different cores for processing, improving program parallelism. Within a single processor core, different processes can also be distributed to multiple processing units simultaneously. Technical issues

[0005] Instruction-level parallelism plays a crucial role in fully utilizing the computational power of multiple arithmetic units and / or multiple processors. However, various dependencies exist between instructions, including architecture dependencies, data dependencies, and control dependencies. Instruction dependencies can easily cause pipeline blockages, limiting the full potential of instruction-level parallelism.

[0006] In view of this, there is an urgent need for an instruction control scheme that can improve instruction-level parallelism, to maximize the potential of instruction parallelism, and to further improve the processor's computing efficiency. Technical solutions

[0007] In order to at least address one or more of the technical problems mentioned above, this disclosure proposes an instruction processing scheme in several aspects.

[0008] In a first aspect, this disclosure provides an instruction processing apparatus, comprising: a decoding unit configured to decode an instruction; a microinstruction generation unit configured to split the decoded instruction into several microinstructions; a microinstruction-level dependency determination unit configured to determine whether there is a microinstruction-level dependency between the currently processed microinstruction and the preceding instruction being executed; and an instruction issuing unit configured to determine whether to issue the currently processed microinstruction based on the microinstruction-level dependency.

[0009] In a second aspect, this disclosure provides an instruction processing method, comprising: decoding an instruction; splitting the decoded instruction into several microinstructions; determining whether there is a microinstruction-level dependency between the currently processed microinstruction and the preceding instruction being executed; and determining whether to issue the currently processed microinstruction based on the microinstruction-level dependency.

[0010] In a third aspect, this disclosure provides a processor including the instruction processing apparatus of the first aspect.

[0011] In a fourth aspect, this disclosure provides a chip including the processor of the aforementioned third aspect.

[0012] In the fifth aspect, this disclosure provides a board including the chip described in the fourth aspect above. Beneficial effects

[0013] Through the instruction processing apparatus, method, processor, chip, and board provided above, the embodiments disclosed herein provide microinstruction-level dependency determination, which can schedule instruction parallel execution in a more granular manner, reduce the latency between dependent instructions, and thereby improve instruction parallelism and processor performance. Attached Figure Description

[0014] The above and other objects, features, and advantages of exemplary embodiments of this disclosure will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this disclosure are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein:

[0015] Figure 1 shows a schematic diagram of the structure of a board according to an embodiment of this disclosure;

[0016] Figure 2 shows a structural diagram of the combined processing device in the chip according to an embodiment of this disclosure;

[0017] Figure 3 shows a schematic diagram of the internal structure of the processor core when the computing device in Figure 2 is a single-core device;

[0018] Figure 4 shows a simplified internal structure diagram of the computing device in Figure 2 when it is multi-core;

[0019] Figure 5A illustrates a schematic diagram of the sequential execution of existing dependent instructions;

[0020] Figure 5B illustrates a schematic diagram of parallel execution between dependent instructions in an embodiment of this disclosure;

[0021] Figure 6 illustrates an instruction execution scenario with various dependencies according to an embodiment of this disclosure;

[0022] Figure 7 shows an exemplary structural block diagram of an instruction processing apparatus according to an embodiment of the present disclosure;

[0023] Figure 8 shows an exemplary structural block diagram of an instruction processing apparatus according to other embodiments of the present disclosure;

[0024] Figure 9 illustrates several possible scenarios between the data memory access addresses of two consecutive instructions;

[0025] Figure 10 shows an exemplary structural block diagram of an instruction processing apparatus according to some embodiments of the present disclosure;

[0026] Figures 11A and 11B exemplarily illustrate schematic diagrams of address comparison optimization schemes according to some embodiments of this disclosure;

[0027] Figures 12A and 12B schematically illustrate examples of implementing a chained parallel mechanism on multidimensional instructions;

[0028] Figure 13 illustrates an exemplary flowchart of an instruction processing method according to an embodiment of this disclosure; and

[0029] Figure 14 shows an exemplary flowchart of an instruction processing method according to other embodiments of this disclosure. Detailed Implementation

[0030] The technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, not all of them. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0031] It should be understood that the terms "first," "second," "third," and "fourth," etc., that may appear in the claims, specification, and drawings of this disclosure are used to distinguish different objects, rather than to describe a specific order. The terms "comprising" and "including" as used in the specification and claims of this disclosure indicate the presence of the described features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or collections thereof.

[0032] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As used in this disclosure and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this disclosure and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0033] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0034] The specific embodiments disclosed herein will now be described in detail with reference to the accompanying drawings.

[0035] Exemplary hardware environment

[0036] Figure 1 shows a schematic diagram of the structure of a board 10 according to an embodiment of this disclosure. As shown in Figure 1, the board 10 includes a chip 101, which is a system-on-a-chip (SoC) that integrates one or more combined processing devices. The combined processing device is an artificial intelligence computing unit used to support various deep learning and machine learning algorithms, meeting the intelligent processing needs of complex scenarios in fields such as computer vision, speech, natural language processing, and data mining. In particular, deep learning technology is widely used in the field of cloud intelligence. A significant characteristic of cloud intelligence applications is the large amount of input data, which places high demands on the platform's storage and computing capabilities. The board 10 of this embodiment is suitable for cloud intelligence applications, possessing massive off-chip storage, on-chip storage, and powerful computing capabilities.

[0037] Chip 101 is connected to external device 103 via external interface device 102. External device 103 may be, for example, a server, computer, camera, monitor, mouse, keyboard, network card, or Wi-Fi interface. Data to be processed can be transmitted from external device 103 to chip 101 via external interface device 102. The calculation results from chip 101 can be transmitted back to external device 103 via external interface device 102. Depending on the application scenario, external interface device 102 may have different interface forms, such as a PCIe interface.

[0038] The board 10 also includes a storage device 104 for storing data, which includes one or more memory cells 105. The storage device 104 is connected to and transmits data with the controller 106 and the chip 101 via a bus. The controller 106 in the board 10 is configured to regulate the state of the chip 101. Therefore, in one application scenario, the controller 106 may include a microcontroller (MCU).

[0039] Figure 2 is a structural diagram illustrating the combined processing device in chip 101 of this embodiment. As shown in Figure 2, the combined processing device 20 includes a computing device 201, an interface device 202, a processing device 203, and a storage device (DRAM) 204.

[0040] The computing device 201 is configured to perform user-specified operations. It is mainly implemented as a single-core intelligent processor or a multi-core intelligent processor to perform deep learning or machine learning calculations. It can interact with the processing device 203 through the interface device 202 to jointly complete the user-specified operations.

[0041] Interface device 202 is used to transmit data and control commands between computing device 201 and processing device 203. For example, computing device 201 can obtain input data from processing device 203 via interface device 202 and write it to on-chip storage device of computing device 201. Further, computing device 201 can obtain control commands from processing device 203 via interface device 202 and write them to on-chip control cache of computing device 201. Alternatively or optionally, interface device 202 can also read data from storage device of computing device 201 and transmit it to processing device 203.

[0042] Processing device 203, as a general-purpose processing device, performs basic control including but not limited to data transfer, and starting and / or stopping computing device 201. Depending on the implementation, processing device 203 may be one or more types of processors, including but not limited to digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, computing device 201 disclosed herein can be considered as having a single-core structure or a homogeneous multi-core structure. However, when computing device 201 and processing device 203 are considered together, they are considered to form a heterogeneous multi-core structure.

[0043] Storage device 204 is used to store data to be processed. It may be DRAM, an off-chip memory, specifically DDR memory, typically 16G or larger, used to store data of computing device 201 and / or processing device 203.

[0044] Figure 3 shows a schematic diagram of the internal structure of the processor core when the computing device 201 in Figure 2 is a single-core device. The computing device 301 is used to process input data such as computer vision, speech, natural language, and data mining. The computing device 301 includes three main modules: a control module 31 (also called a controller), an arithmetic module 32 (also called an arithmetic unit), and a storage module 33 (also called a memory).

[0045] The control module 31 coordinates and controls the operation of the computation module 32 and the storage module 33 to complete the deep learning task. It includes an instruction fetch unit (IFU) 311 and an instruction decode unit (IDU) 312. The instruction fetch unit 311 fetches instructions from the processing device 203, and the instruction decode unit 312 decodes the fetched instructions and sends the decoding result as control information to the computation module 32 and the storage module 33.

[0046] The computation module 32 includes a vector operation unit 321 and a matrix operation unit 322. The vector operation unit 321 is used to perform vector operations and can support complex operations such as vector multiplication, addition, and nonlinear transformations; the matrix operation unit 322 is responsible for the core computations of deep learning algorithms, namely matrix multiplication and convolution.

[0047] Storage module 33 is used to store or move relevant data, including neuron RAM (NRAM) 331, weight RAM (WRAM) 332, and direct memory access (DMA) module 333. NRAM 331 stores input neurons, output neurons, and intermediate results after computation; WRAM 332 stores the convolution kernels of the deep learning network, i.e., the weights; DMA 333 is connected to DRAM 204 via bus 34 and is responsible for data transfer between computing device 301 and DRAM 204. It should be noted that NRAM and WRAM here can be two storage areas formed by dividing the same memory in logical storage space, or they can be two independent memories; no specific limitation is made here.

[0048] Figure 4 shows a simplified internal structure of the computing device 201 in Figure 2 when it is a multi-core system. Multi-core computing devices can be abstracted using a hierarchical hardware model. As shown, the multi-core computing device 400 is a system-on-a-chip that includes at least one computing cluster, and each computing cluster includes multiple processor cores. In other words, the multi-core computing device 400 is constructed in a hierarchical structure of system-on-a-chip, computing cluster, and processor cores.

[0049] From the perspective of the system-on-a-chip hierarchy, as shown in the figure, the multi-core computing device 400 includes an external memory controller 41, a peripheral communication module 42, an on-chip interconnect module 43, a global synchronization module 44, and multiple computing clusters 45.

[0050] There can be multiple external storage controllers 41; two are shown exemplarily in the figure. These controllers respond to access requests from the processor core to access external storage devices (e.g., DRAM 204 in Figure 2), thereby reading data from or writing data to external storage. The peripheral communication module 42 receives control signals from the processing device (203 in Figure 2) via an interface device (202 in Figure 2) and initiates the computing device (201 in Figure 2) to perform tasks. The on-chip interconnect module 43 connects the external storage controllers 41, the peripheral communication module 42, and the multiple computing clusters 45 to transmit data and control signals between the modules. The global synchronization module 44 is, for example, a global barrier controller (GBC) to coordinate the working progress of each computing cluster and ensure information synchronization. The multiple computing clusters 45 are the computing cores of the multi-core computing device 400; four are shown exemplarily on each die in the figure. With hardware development, the multi-core computing device 400 disclosed herein may also include eight, sixteen, sixty-four, or even more computing clusters 45. Computation cluster 45 is used to efficiently execute deep learning algorithms.

[0051] From the perspective of the computing cluster hierarchy, as shown in the figure, each computing cluster 45 includes multiple processor cores 406 as control and computing units, and also a shared memory core 407 as a storage unit. Furthermore, each computing cluster may also include a local synchronization module 412 to coordinate the working progress of each processor core within the computing cluster, ensuring information synchronization. Four processor cores 406 are exemplarily shown in the figure; this disclosure does not limit the number of processor cores 406.

[0052] The storage core 407 is primarily used for storage and communication, namely storing shared data or intermediate results among processor cores 406, and performing communication between computing clusters 45 and DRAM 204, communication between computing clusters 45, and communication between processor cores 406. In other embodiments, the storage core 407 has scalar operation capabilities and is used to perform scalar operations.

[0053] Storage core 407 includes a shared memory unit (SMEM) 408, a broadcast bus 409, a cluster direct memory access (CDMA) module 410, and a global direct memory access (GDMA) module 411. SMEM 408 acts as a high-performance data relay station. Data multiplexed between different processor cores 406 within the same compute cluster 45 does not need to be obtained from the DRAM 204 by each processor core 406 individually. Instead, it is relayed between processor cores 406 via SMEM 408. Storage core 407 only needs to quickly distribute the multiplexed data from SMEM 408 to multiple processor cores 406, thereby improving inter-core communication efficiency and significantly reducing on-chip and off-chip I / O access. Broadcast bus 409, CDMA 410, and GDMA 411 are used to perform communication between processor cores 406, communication between compute clusters 45, and data transfer between compute clusters 45 and DRAM 204, respectively.

[0054] From the perspective of processor cores, the structure of a single processor core can be similar to the structure diagram of a single-core computing device shown in Figure 3, which will not be described in detail here.

[0055] Instruction processing scheme

[0056] As mentioned earlier, dependencies between instructions can easily cause pipeline blockages, limiting the advantages of instruction-level parallelism. In most instruction set architectures (ISAs), dependencies between instructions can be determined by judging the address ranges of instruction reads and writes. The hardware preprocesses the instructions to calculate the address ranges of each operand, thus determining which instructions can be executed in parallel.

[0057] For example, for two instructions S1 and S2, if the address ranges of the write operands of S1 and arbitrary operands of S2 overlap, or the address ranges of the read operands of S1 and write operands of S2 overlap, then S1 and S2 are considered to have a dependency relationship. That is, if there is any overlap in the operand ranges between two instructions, and at least one of the overlapping operands is a write operand, then there is a dependency between these two instructions.

[0058] Once the dependencies between instructions are identified, the hardware can decide to execute the instructions in parallel, ensuring sequential consistency. That is, if an instruction has no dependency on any earlier, incomplete instruction, it can be issued immediately. From the programmer's perspective, the execution result is consistent with serial execution. When the hardware can determine dependencies between instructions and guarantee correctness, because all instructions are ordered, instructions do not need to be distinguished by their instruction stream, and synchronization instructions within the kernel can be omitted.

[0059] However, the above method only allows for parallel execution of completely independent instructions. If two instructions are dependent on each other, the next instruction can only be issued after the previous instruction has completed and committed. This results in both excessively granular instruction parallel scheduling and high latency between dependent instructions.

[0060] In view of this, the present disclosure provides a microinstruction-level dependency determination scheme. The basic idea is to break down instructions into microinstructions (uops) and then determine dependencies based on these microinstructions. When there are instruction-level dependencies, compared to serial execution on an instruction-by-instruction basis, this method can significantly reduce latency between dependent instructions and achieve more granular parallelism between instructions.

[0061] Figure 5A illustrates a schematic diagram of the serial execution of existing dependent instructions; Figure 5B illustrates a schematic diagram of the parallel execution of dependent instructions according to an embodiment of this disclosure. In the figures, the horizontal axis represents time, and the vertical axis represents four different memory access spaces A0 to A3. The figures show two computation instructions, where instruction C1 involves writing data to the four memory spaces A0 to A3, and instruction C2 involves reading data from the four memory spaces A0 to A3.

[0062] As shown in Figure 5A, according to the existing instruction parallelism scheme, due to the dependency between instructions C1 and C2, the subsequent instruction C2 can only be executed after instruction C1 has completed execution and the dependency has been removed. Specifically, since instruction C1 has no dependency with the preceding instructions, instruction C1 can be executed, sequentially writing data to memory access spaces A0 to A3. After writing to A3 is completed, the dependency of instruction C2 is removed, and then instruction C2 is executed, sequentially reading data from memory access spaces A0 to A3. Therefore, it is evident that when instructions C1 and C2 have a dependency, they can only be executed serially.

[0063] In contrast, as shown in Figure 5B, according to the instruction parallelism scheme of this embodiment, even if there is a dependency between instructions C1 and C2, instructions C1 and C2 are broken down into several micro-instructions. These micro-instructions can be executed in parallel without dependency, without waiting for the entire instruction C1 to be executed. Specifically, since instruction C1 has no dependency with its preceding instruction, instruction C1 can be executed. At this time, all four micro-instructions that instruction C1 is broken down into can be executed, that is, data is written to memory access spaces A0 to A3 in sequence. For instruction C2, since there is a dependency between instruction C2 and instruction C1, the dependency of the micro-instructions of instruction C2 can be determined. For example, the micro-instruction of instruction C2 that reads space A0 has a dependency with the micro-instruction of instruction C1 that writes space A0. However, after the micro-instruction of writing space A0 is executed, the dependency between the two is removed, and the micro-instruction of instruction C2 that reads space A0 can be executed. The remaining micro-instructions of instruction C2 can be similarly determined in terms of dependency, and the blocking and releasing of micro-instructions can be performed according to the dependency. Therefore, even though instructions C1 and C2 have a dependency relationship, a certain degree of parallel execution of instructions can still be achieved by judging the dependency relationship at the microinstruction level. As shown in Figure 5B, while executing the microinstruction for writing to space A1 of instruction C1, the microinstruction for reading to space A0 of instruction C2 can be executed simultaneously; while executing the microinstruction for writing to space A2 of instruction C1, the microinstruction for reading to space A1 of instruction C2 can be executed simultaneously; and so on. As can be seen from the figure, this parallel method interweaves the microinstructions of different instructions like a chain, and this execution method is called "chaining" in this paper. In this way, instructions C1 and C2, which have a dependency relationship, can still achieve parallelism at the microinstruction level, thereby greatly reducing the latency between instructions.

[0064] The example above describes a scenario where there is a write-after-read dependency between instructions. In addition to write-after-read dependencies, the "chained" execution mechanism provided in this disclosure embodiment can also be applied to scenarios with read-after-write and write-after-write dependencies, avoiding performance loss caused by latency during instruction switching.

[0065] Figure 6 illustrates an instruction execution scenario with various dependencies according to an embodiment of this disclosure. In the figure, the horizontal axis represents time, and the vertical axis represents four different memory access spaces, A0 to A3.

[0066] The diagram illustrates three instructions, in order: Load instruction L1 loads data from external storage into local memory spaces A0-A3; Calculation instruction C1 performs calculations on the data in memory spaces A0-A3; and Load instruction L2 loads data from external storage into local memory spaces A0-A3. That is, after the data loaded by instruction L1 is used by calculation instruction C1, the data is loaded into the same location a second time by instruction L2. At this point, there is a write-after-read dependency between calculation instruction C1 and load instruction L1, a read-after-write dependency between load instruction L2 and calculation instruction C1, and a write-after-write dependency between load instruction L2 and load instruction L1.

[0067] Through the "chained" execution mechanism of this disclosed embodiment, instruction C1 can be executed without waiting for the execution of its dependent instruction L1, and instruction L2 can be executed without waiting for the execution of its dependent instructions L1 and C1. This allows for a continuous flow of data to the storage space even when the Load instruction experiences a certain delay due to the need to first issue a read request. For example, as can be seen from the entire timeline, the data written to the storage space is continuously provided; that is, the light gray shaded area in the figure is continuous on the timeline.

[0068] Figure 7 shows an exemplary structural block diagram of an instruction processing apparatus according to an embodiment of the present disclosure. As shown, the instruction processing apparatus 700 includes a decoding unit 710, a microinstruction generation unit 720, a microinstruction-level dependency determination unit 730, and an instruction issuing unit 740.

[0069] The decoding unit 710 is configured to decode instructions. The primary task of the decoding unit is to translate instructions (i.e., machine code) fetched from memory into internal signals that the processor can understand. These signals instruct various parts of the processor how to execute the instructions. Specifically, the decoding unit identifies the opcode in the instruction, which is the part of the instruction used to identify the type of operation to be performed. Based on the opcode, the decoding unit determines whether the instruction is an arithmetic operation, logical operation, data transfer, or control flow operation, etc. The decoding unit also parses the operands in the instruction, including the source operand and the destination operand. It determines the operand type (such as immediate value, register, or memory address) and addressing mode (such as direct addressing, indirect addressing, etc.). The decoding unit also generates control signals that are passed to other parts of the processor, such as the arithmetic logic unit (ALU), data storage unit, and control unit, to guide them in executing the operations required by the instructions.

[0070] The decoding unit typically includes an instruction buffer queue for temporary storage of instructions, which are then fed into the decoding unit at a certain rate (e.g., 6 instructions per cycle) for decoding. The decoded instructions are then passed to the next pipeline stage.

[0071] The microinstruction generation unit 720 is configured to break down the decoded instructions into several microinstructions. The primary task of the microinstruction generation unit is to decompose complex machine instructions into a series of simpler microinstructions. These microinstructions are the basic steps to implement specific operations and typically correspond to basic operations within the processor.

[0072] The microinstruction generation unit can split instructions into several microinstructions according to a predetermined splitting strategy. Typically, splitting is done according to the basic operations that implement the instructions. For cases with large operand data volumes, further splitting can be performed based on the data itself. For example, when the operands are long vectors, they can be split into microinstructions that operate on multiple short vectors.

[0073] The microinstruction-level dependency determination unit 730 is configured to determine whether there is a microinstruction-level dependency between the currently processed microinstruction and the preceding instruction being executed. This dependency mainly refers to data dependencies, such as write-after-read dependency, write-after-write dependency, and read-after-write dependency. Therefore, the determination of dependencies is primarily based on the memory access addresses of the data.

[0074] In some embodiments, for any microinstruction involving a write operation, if the data memory access address of the currently processed microinstruction does not overlap with the data memory access addresses of all incomplete microinstructions in the preceding instructions, it can be determined that there is no microinstruction-level dependency between the currently processed microinstruction and the preceding instructions. Conversely, if there is overlap, it indicates that a microinstruction-level dependency exists. Here, "any microinstruction involving a write operation" refers to either of the two parts being compared, encompassing the three possible scenarios of dependency relationships described above.

[0075] The instruction issuing unit 740 is configured to determine whether to issue the currently processed microinstruction based on microinstruction-level dependencies. If it is determined that there is no microinstruction-level dependency between the current microinstruction and its preceding instructions, the instruction issuing unit can issue the microinstruction to a subsequent execution unit for execution. If it is determined that there is a microinstruction-level dependency between the current microinstruction and any preceding instruction, the instruction issuing unit can block the microinstruction until the microinstruction-level dependency is resolved.

[0076] By setting microinstruction-level dependency judgments, instruction parallel execution can be scheduled more finely, reducing latency between dependent instructions and thus improving instruction parallelism and processor performance.

[0077] Figure 8 illustrates an exemplary structural block diagram of an instruction processing apparatus according to other embodiments of this disclosure. In these embodiments, the instruction processing apparatus 800 includes, in addition to a decoding unit 810, a microinstruction generation unit 820, a microinstruction-level dependency determination unit 830, and an instruction issuing unit 840, an instruction-level dependency determination unit 850.

[0078] The instruction-level dependency determination unit 850 can be configured to determine whether an instruction-level dependency exists between the currently processed instruction and its preceding execution instruction. In this case, the micro-instruction-level dependency determination unit 830 can be further configured to: when the instruction-level dependency determination unit determines that an instruction-level dependency exists, further determine whether a micro-instruction-level dependency exists. Therefore, the instruction issuing unit 840 can be further configured to: when neither the instruction-level dependency nor the micro-instruction-level dependency indicates a dependency, issue the currently processed micro-instruction. It is understood that the arrows in the figure are merely illustrative and do not represent that all embodiments are connected in this way. For example, although the figure shows a connection between the instruction-level dependency determination unit 850 and the instruction issuing unit 840, it does not mean that the two need to be directly connected; the information can also be transmitted to the micro-instruction-level dependency determination unit 830 and then to the instruction issuing unit 840. For example, the instruction-level dependency determination unit 850 can pass information indicating no instruction-level dependency to the micro-instruction-level dependency determination unit 830, and the micro-instruction-level dependency determination unit 830 can determine that there is definitely no micro-instruction-level dependency based on this, thereby instructing the instruction issuing unit 840.

[0079] By first determining instruction-level dependencies and then only determining micro-instruction-level dependencies for instructions with instruction-level dependencies, the overhead of comparison and judgment can be effectively reduced, thereby improving processor efficiency.

[0080] Determining dependencies depends on whether there is overlap between data memory access addresses. In principle, determining microinstruction-level dependencies involves comparing the data memory access address of the subsequent microinstruction with the data memory access addresses of all incomplete microinstructions in the preceding instructions. To reduce comparison overhead, a dynamic instruction range can be recorded for each instruction, representing the range of addresses accessed by the incomplete portion of the instruction, and updated immediately after each microinstruction commit.

[0081] Therefore, in some embodiments, the instruction processing apparatus 800 may further include an address range maintenance unit 860, configured to maintain a dynamic address range for each preceding instruction being executed. This address range includes the start and end addresses of the data memory access addresses corresponding to the incomplete portion of the preceding instruction. Further, the address range maintenance unit may be further configured to update the aforementioned address range of its corresponding preceding instruction in response to the submission of a microinstruction. For sequentially executed instructions, this instruction range gradually decreases, thereby freeing up address space and allowing subsequent instructions' microinstructions to change from dependent to independent, thus enabling execution. This real-time updating of the address range of the incomplete portions of each instruction facilitates the determination of dependencies, timely removal of dependencies, and release of microinstructions.

[0082] Because instructions exhibit sequential execution characteristics in the "chained" data memory access dimension, such as execution from front to back (as described later in the context of multidimensional instructions), microinstruction-level dependency determination does not require considering the end address of preceding instructions; only the dynamically changing start address of the preceding instructions needs to be considered. Accordingly, the microinstruction of the current instruction can be determined by comparing its end address with the start address of the preceding instructions. If the end address is less than the start address of the incomplete portions of all preceding instructions, it indicates no dependency and can be issued. More specifically, microinstruction-level dependencies can be determined by comparing the end address of the current instruction's microinstruction with the minimum value (limit) of the current start addresses of all preceding instructions.

[0083] When both instruction-level dependency judgment and microinstruction-level dependency judgment exist, the judgment method can be further optimized to reduce the overhead of comparison judgment.

[0084] In some embodiments, the instruction-level dependency determination unit 850 may be further configured to: for any instruction involving a write operation, if the starting address in the data memory access address of the currently processed instruction is greater than the current ending address of the preceding instruction being executed, determine that there is no instruction-level dependency; otherwise, determine that there is an instruction-level dependency.

[0085] In these embodiments, the microinstruction-level dependency determination unit 830 may be further configured to: for a microinstruction with an instruction-level dependency, if the end address in the data memory access address of the microinstruction is less than the current start address of the preceding instruction being executed, determine that there is no microinstruction-level dependency; otherwise, determine that there is a microinstruction-level dependency.

[0086] Furthermore, the microinstruction-level dependency determination unit 830 can be further configured to: for a microinstruction with an instruction-level dependency, if the end address in the data memory access address of the microinstruction is less than the minimum value limit of the current start address of all preceding instructions with which it has an instruction-level dependency, determine that there is no microinstruction-level dependency.

[0087] The following section will explain the logic for determining the above dependency relationship in more detail by combining several scenarios.

[0088] Figure 9 illustrates several possible scenarios between the data memory access addresses of two consecutive instructions. In the figure, the light-colored box represents the preceding instruction inst0, and the dark-colored box represents the following instruction inst1. The boxes represent data memory access spaces, defined by the start and end addresses, respectively. For ease of observation, the boxes representing data memory access spaces are slightly offset vertically. It can be intuitively seen from Figure 9 that there is no dependency between the two instructions inst0 and inst1 in scenarios A and D, while there is a dependency in scenarios B and C. When applying the "chained" execution of the embodiments disclosed herein:

[0089] In scenario A, if we follow the rule of judging microinstruction-level dependencies, that is, there is no dependency when the end address of the current instruction's microinstruction is less than the current start address of all preceding instructions, then since the start address of instruction inst0_start is always relatively small, inst1 will always be considered to have a dependency and will be blocked until inst0 finishes execution, thus creating a false dependency.

[0090] In scenario B, initially, according to the microinstruction dependency rules, `inst1` is blocked because the end address of the first few microinstructions is greater than the start address of `inst0`. As the microinstructions of `inst0` are executed, their start address `inst0_start` increments, gradually exceeding the end address of the first few microinstructions of `inst1`. Therefore, the first few microinstructions of `inst1` can be released and execution can begin, and so on. When `inst0` completes, it is committed, at which point the remaining microinstructions of `inst1` can be released.

[0091] In scenario C, initially, according to the microinstruction-level dependency judgment rules, since the end address of the microinstructions preceding inst1 is less than the start address of instruction inst0, inst1 can execute some microinstructions. When execution reaches the overlapping part with inst0, inst1 is blocked. At this time, it waits for the start address of inst0, inst0_start, to increment, and as the start address of inst0 increments, the dependency is gradually released until execution is complete.

[0092] In case D, according to the rules for judging microinstruction-level dependencies, the end address of the microinstruction of instruction inst1 is always less than the start address of instruction inst0_start, so it is considered to have no dependency and the entire instruction inst1 can be executed.

[0093] The case where the instructions inst0 and inst1 contain each other can be regarded as a combination of case B and case C. The execution process is similar and will not be elaborated here.

[0094] As can be seen from the above scenarios, in the chained execution phase based on microinstruction-level dependency judgment, scenarios B, C, and D can all achieve chained execution normally, while scenario A will result in false dependencies. Therefore, when microinstruction-level dependency judgment and instruction-level dependency judgment exist simultaneously, scenario A can be judged through instruction-level dependency judgment to eliminate false dependencies, while other scenarios can be handled through microinstruction-level dependency judgment.

[0095] Therefore, when determining instruction-level dependencies, for any instruction involving a write operation, if the starting address of the data memory access address of the currently processed instruction is greater than the current ending address of the preceding instruction, it is determined that there is no instruction-level dependency; otherwise, it is determined that there is an instruction-level dependency. Specifically, for the above four scenarios, the instruction-level dependency determination will consider A to have no dependency, while B, C, and D will all have dependencies.

[0096] For instructions identified as having instruction-level dependencies, a micro-instruction-level dependency determination is further performed. At this point, according to the above analysis, in the chained execution phase based on micro-instruction-level dependency determination, all three scenarios (B, C, and D) can achieve pipelined execution normally.

[0097] The above describes the dependency determination and chained execution process from the perspective of processing a single instruction or microinstruction. In real-world scenarios, multiple instructions are typically processed simultaneously, with each instruction broken down into multiple microinstructions, resulting in multiple instructions and microinstructions being processed in parallel.

[0098] Figure 10 illustrates an exemplary structural block diagram of an instruction processing apparatus according to some embodiments of the present disclosure. In these embodiments, the instruction processing apparatus 1000 may include a launch queue 1010 and a plurality of launch slots 1020.

[0099] The issue queue 1010 supports instruction-level dependency determination and can select instructions for execution out of order. Specifically, the issue queue may include a decoding unit 1011, an instruction-level dependency determination unit 1012, and an address range maintenance unit 1013. The functions and implementations of these components can be referred to the previous description and will not be repeated here.

[0100] The issue slot 1020 can be used to store instructions and microinstructions currently being decoded and executed in parallel, and to perform microinstruction-level dependency judgment. Specifically, the issue slot 1020 may include a microinstruction generation unit 1021, a microinstruction-level dependency judgment unit 1022, an instruction issue unit 1023, and a microinstruction submission queue 1024. The microinstruction submission queue 1024 is used to respond to the execution unit (not shown) generating microinstruction submission information and perform corresponding submission processing, such as sending it to the address range maintenance unit 1013 to update the address range of the instructions being executed. The functions and implementations of other components can be referred to the previous description, and will not be repeated here.

[0101] The instruction processing device 1000 may include multiple issue slots 1020. The specific implementation details of the issue slots may differ slightly depending on the type of instruction, but they will generally include the aforementioned components. Multiple issue slots can exist for the same type of instruction. Instructions cached in different issue slots are independent of each other, and instructions within the same issue slot are issued in an ordered manner.

[0102] In the chained execution phase based on microinstruction-level dependency judgment, the process of comparing the minimum value of the starting addresses of all instruction operands requires a large number of address comparison operations, which consumes a lot of hardware and time resources, especially when there are a large number of instructions and / or microinstructions being executed at the same time.

[0103] In some embodiments disclosed herein, the address comparison process can be optimized by leveraging the characteristic that all microinstruction-level dependency judgment units share the address range information of the preceding instruction, thereby reducing hardware and time consumption.

[0104] Specifically, the aforementioned address range maintenance unit 1013 can be further configured to: compare the current start addresses of all preceding instructions being executed pairwise to obtain a comparison matrix. This comparison matrix can then be transmitted to each microinstruction-level dependency determination unit 1022. Each microinstruction-level dependency determination unit 1022 can then be further configured to: based on the comparison matrix, and using the instruction-level dependencies of the currently processed microinstruction, determine the minimum value of the current start addresses of all preceding instructions that have instruction-level dependencies with the currently processed microinstruction. Then, the end address of the currently processed microinstruction can be compared with the minimum value of this start address to determine whether a microinstruction-level dependency exists.

[0105] In other words, a complete address comparison can be performed first, that is, comparing the current starting addresses of any two instructions among all incomplete preceding instructions to obtain a comparison matrix. Then, the comparison matrix is ​​broadcast to each microinstruction-level dependency determination unit. Each microinstruction-level dependency determination unit can select the minimum value of the instruction it depends on from the comparison matrix according to its own instruction-level dependency to compare and determine the microinstruction-level dependency.

[0106] Figures 11A and 11B illustrate schematic diagrams of address comparison optimization schemes according to some embodiments of this disclosure.

[0107] Figure 11A shows the complete comparison matrix obtained by comparing the current starting addresses of all preceding instructions in execution pairwise. In this example, assuming there are four preceding instructions in execution, with instruction numbers 1, 3, 5, and 2, the comparison matrix is ​​a 4×4 matrix, where each element represents the comparison result of the current starting address of the instruction at the corresponding row and column coordinates. For example, if the current starting address of the row coordinate instruction is less than the current starting address of the column coordinate instruction, the matrix element at that location is 1; otherwise, it is 0. The elements on the diagonal of the matrix are considered invalid. Therefore, the column with all valid values ​​of 1 represents the instruction containing the minimum value, or conversely, the row with all valid values ​​of 0 represents the instruction containing the minimum value. For example, in Figure 11A, instruction 5 has the smallest current starting address.

[0108] Figure 11B shows a schematic diagram of the microinstruction-level dependency judgment unit extracting the minimum value it needs from the comparison matrix.

[0109] Each microinstruction-level dependency determination unit can determine the instruction-level dependencies of the current microinstruction. Specifically, instruction-level dependencies can be represented using a bitmap, where each bit indicates whether there is a dependency between the current instruction and a preceding instruction in execution. The determination of instruction-level dependencies can be referred to the previous description. For example, if the current instruction has no dependency on the first preceding instruction, the first bit in the bitmap can be set to "1"; if the current instruction has a dependency on the second preceding instruction, the second bit in the bitmap can be set to "0"; and vice versa.

[0110] Therefore, the microinstruction-level dependency determination unit can use this bitmap as a mask and apply it to the comparison matrix to select the minimum starting address of the instruction it depends on. For example, as shown in Figure 11B, assuming the preceding instructions with which the current microinstruction has a dependency are instruction 3 and instruction 2, and the bitmap is 0101 (NYNY in the figure), then the column with all valid values ​​of 1 is selected from the columns of instruction 3 and instruction 2, that is, the column of instruction 3. That is, among the preceding instructions with which the current microinstruction has a dependency, the current starting address of instruction 3 is the smallest. Subsequently, the microinstruction-level dependency determination unit can use the current starting address of instruction 3 to compare with the ending address of the current microinstruction to determine whether a microinstruction-level dependency exists.

[0111] In this way, when the number of microinstructions processed in parallel is large, comparison time and hardware resources can be effectively saved.

[0112] Artificial intelligence processors typically process high-dimensional tensor data. For multidimensional instructions with operands consisting of multidimensional data blocks, there are often many microinstruction splitting strategies. Therefore, different splitting strategies for consecutive instructions can lead to different dimensions covered by the microinstructions of different instructions. However, the "chained" parallel mechanism of the embodiments disclosed in this paper can still be applied to a certain extent.

[0113] In some embodiments, when the operand of the current instruction is a multidimensional data block, and the operand of a preceding instruction with an instruction-level dependency is also the same multidimensional data block, a "chained" parallel mechanism can be implemented on the dimension where the access order of the current instruction and the preceding instruction is consistent. In this case, the microinstruction-level dependency determination unit can be further configured to: perform the aforementioned microinstruction-level dependency determination on the dimension where the access order of the current instruction and the preceding instruction is consistent. That is, when determining the microinstruction-level dependency, the comparison is based on the address range of the data on the dimension where the access order is consistent.

[0114] Figures 12A and 12B schematically illustrate examples of implementing a chained parallel mechanism on multidimensional instructions.

[0115] Figure 12A illustrates a scenario where the preceding instruction is a Load instruction, followed by a Conv instruction. The three data spaces on the left (a), (b), and (c) show the changes in data loaded during the execution of the Load instruction over time, while the three data spaces on the right (d), (e), and (f) show the data used during the execution of the Conv instruction over time. The data blocks in the figure are three-dimensional, including dimensions C, W, and H. Dimension C is the lowest dimension, meaning data in dimension C is stored contiguously in the one-dimensional storage space. Dimension W is the intermediate dimension, and dimension H is the highest dimension.

[0116] The micro-instruction splitting strategy of the Load instruction is to fetch data from the layer above each time, for example, a layer of W×C data. As shown in Figure (a), initially, a light gray data block is loaded into the data space. As the micro-instruction is executed, Figure (b) shows that 4 layers of W×C data have been loaded; Figure (c) shows that 12 layers of W×C data have been loaded.

[0117] The micro-instruction splitting strategy of the Conv instruction is to fetch 4×4×4 (H×W×C) data at a time. Therefore, when the Load instruction executes to load one layer of W×C data, the micro-instructions of the Conv instruction cannot be executed, as shown in Figure (d), and must continue to wait. When the Load instruction executes to load 4 layers of W×C data, the micro-instructions of the Conv instruction can be executed. As shown in Figure (e), the Conv instruction can split the 4×W×C data block into multiple 4×4×4 data blocks, executing the corresponding micro-instructions in the order of C dimension first, then W dimension. And so on, when the Load instruction executes to load 12 layers of W×C data, the Conv instruction can execute the micro-instructions of the corresponding third layer of 4×W×C data blocks, as shown in Figure (f).

[0118] Therefore, although the micro-instruction splitting strategies of Load and Conv instructions are different, from the H dimension, their processing / access order is consistent, both being processed from top to bottom. Thus, a "chained" parallel mechanism can be implemented at the H dimension.

[0119] Figure 12B illustrates a scenario where the preceding instruction is a matrix multiplication Matmul instruction, followed by a vector Vec instruction. The three data spaces on the left (a), (b), and (c) show the changes in data stored within the Matmul instruction over time, while the three data spaces on the right (d), (e), and (f) show the data used by the Vec instruction over time. The data blocks in the figure are two-dimensional, comprising M and N dimensions, where N is the lowest dimension and M is the highest dimension.

[0120] The Matmul instruction's micro-instruction splitting strategy is to compute a small matrix block at a time, for example, a matrix of size 2×N / 2. As shown in Figure (a), initially, a light gray matrix block is computed and stored in the data space. As the micro-instruction is executed, Figure (b) shows that two light gray matrix blocks have been computed; Figure (c) shows that six light gray matrix blocks have been computed.

[0121] The Vec instruction's microinstruction splitting strategy is to fetch one row of data at a time. Therefore, when the Matmul instruction only calculates one light gray matrix block, the Vec instruction's microinstructions cannot be executed, as shown in Figure (d), and it must continue to wait. Once the Matmul instruction has calculated two light gray matrix blocks, i.e., two complete rows of data, the Vec instruction's microinstructions can be executed. As shown in Figure (e), the Vec instruction can split the two light gray matrix blocks into two rows of data and execute the corresponding microinstructions row by row. Similarly, when the Matmul instruction has calculated six light gray matrix blocks, the Vec instruction can execute the microinstructions corresponding to the 5th and 6th rows of data, as shown in Figure (f).

[0122] Therefore, although the micro-instruction decomposition strategies of the Matmul and Vec instructions are different, their processing / access order is consistent from the M-dimensional perspective, both being processed from top to bottom. Thus, a "chained" parallel mechanism can be implemented in the M-dimensional space.

[0123] Optionally or additionally, in some embodiments, the microinstruction generation unit may be further configured to: adjust the splitting strategy of the current instruction so that there is at least one dimension with consistent access order between the split microinstruction and the preceding instruction. In some implementations, when there is no dimension with consistent access order between the split microinstruction and the preceding instruction according to the current instruction splitting strategy, the current instruction splitting strategy can be adjusted so that there is at least one dimension with consistent access order between the split microinstruction and the preceding instruction, thereby enabling a "chained" parallel mechanism to be implemented on that dimension. In other implementations, even if there is a dimension with consistent access order between the split microinstruction and the preceding instruction according to the current instruction splitting strategy, the current instruction splitting strategy can be adjusted so that there are more dimensions with consistent access order between the split microinstruction and the preceding instruction, thereby enabling a "chained" parallel mechanism to be implemented on a larger scale. For example, in the example shown in Figure 12A, if possible, the splitting strategy of the Conv instruction can be adjusted to be consistent with the splitting strategy of the Load instruction. Then, as the Load instruction completes the execution of each microinstruction (loading a layer of data), the corresponding Conv instruction's microinstruction (performing a convolution operation on this layer of data) can be executed.

[0124] Figure 13 shows an exemplary flowchart of an instruction processing method 1300 according to an embodiment of this disclosure.

[0125] As shown in the figure, in step 1310, the instruction is decoded; then, in step 1320, the decoded instruction is broken down into several micro-instructions; in step 1330, it is determined whether there is a micro-instruction-level dependency between the currently processed micro-instruction and the preceding instruction being executed; and in step 1340, based on the determined micro-instruction-level dependency, it is determined whether to issue the currently processed micro-instruction.

[0126] In some embodiments, determining whether there is a microinstruction-level dependency between the currently processed microinstruction and the preceding instruction being executed may further include: for any microinstruction involving a write operation, if the data memory access address of the currently processed microinstruction does not overlap with the data memory access addresses of all incomplete microinstructions in the preceding instruction, it is determined that there is no microinstruction-level dependency between the currently processed microinstruction and the preceding instruction.

[0127] Figure 14 shows an exemplary flowchart of an instruction processing method 1400 according to other embodiments of this disclosure.

[0128] As shown in the figure, in step 1410, the instruction is decoded; then, in step 1420, it is determined whether there is an instruction-level dependency between the currently processed instruction and the preceding instruction being executed; in step 1430, the decoded instruction is broken down into several micro-instructions; in step 1440, when it is determined that there is an instruction-level dependency, it is further determined whether there is a micro-instruction-level dependency between the currently processed micro-instruction and the preceding instruction being executed; and in step 1450, when there is no dependency between any indication of the instruction-level dependency and the micro-instruction-level dependency, the currently processed micro-instruction is emitted.

[0129] In some embodiments, the instruction processing method 1400 may further include step 1460, updating the address range of the corresponding preceding instruction in response to the commit of a microinstruction. A dynamic address range can be maintained for each preceding instruction in execution, including the start and end addresses of the data memory access addresses corresponding to the incomplete portion of the preceding instruction. This address range can be updated accordingly whenever the microinstruction is completed and committed.

[0130] In some embodiments, determining whether there is an instruction-level dependency between the currently processed instruction and the preceding instruction being executed may include: for any instruction involving a write operation, if the starting address of the data memory access address of the currently processed instruction is greater than the current ending address of the preceding instruction being executed, it is determined that there is no instruction-level dependency; otherwise, it is determined that there is an instruction-level dependency.

[0131] In some embodiments, determining whether a microinstruction-level dependency exists may include: for a microinstruction with an instruction-level dependency, if the end address in the data memory access address of the microinstruction is less than the current start address of the preceding instruction being executed, it is determined that no microinstruction-level dependency exists; otherwise, it is determined that a microinstruction-level dependency exists.

[0132] In some embodiments, determining whether the microinstruction-level dependency exists may further include: for a microinstruction with an instruction-level dependency, if the end address in the data memory access address of the microinstruction is less than the minimum value limit of the current start address of all preceding instructions that have an instruction-level dependency with it, then it is determined that there is no microinstruction-level dependency.

[0133] In some embodiments, the instruction processing method 1400 may further include: performing pairwise comparisons of the current starting addresses of all preceding instructions being executed to obtain a comparison matrix; and, based on the comparison matrix, using the instruction-level dependencies of the currently processed microinstruction, determining the minimum value of the current starting addresses of all preceding instructions that have instruction-level dependencies with the currently processed microinstruction.

[0134] In some embodiments, when the operand of the current instruction is a multidimensional data block, and the operand of the preceding instruction that has an instruction-level dependency relationship with it is also the same multidimensional data block, determining whether there is a microinstruction-level dependency relationship between the currently processed microinstruction and the executing preceding instruction may include: performing a microinstruction-level dependency relationship judgment on the dimension where the access order of the current instruction and the preceding instruction is consistent.

[0135] Optionally or additionally, in some embodiments, the instruction processing method 1400 may further include: adjusting the splitting strategy of the current instruction so that there is at least one dimension of consistent access order between the split microinstruction and the preceding instruction.

[0136] Those skilled in the art will understand that the various features of the instruction processing apparatus described above in conjunction with the accompanying drawings can be similarly applied to the instruction processing methods of Figures 13 and 14, and therefore will not be repeated here.

[0137] This disclosure also provides a processor, including the aforementioned instruction processing apparatus for implementing the instruction processing method. This disclosure further provides a chip, which may include the processor of any of the embodiments described above in conjunction with the accompanying drawings. Furthermore, this disclosure also provides a board that may include the aforementioned chip.

[0138] Depending on the application scenario, the electronic devices or apparatus disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablets, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, dashcams, navigators, sensors, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, autonomous driving terminals, vehicles, home appliances, and / or medical devices. The vehicles include airplanes, ships, and / or vehicles; the home appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, lights, gas stoves, and range hoods; the medical devices include MRI scanners, ultrasound machines, and / or electrocardiographs. The electronic devices or apparatus disclosed herein can also be applied in fields such as the Internet, IoT, data centers, energy, transportation, public management, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and healthcare. Furthermore, the electronic devices or apparatus disclosed herein can also be used in application scenarios related to artificial intelligence, big data, and / or cloud computing, such as cloud computing, edge computing, and terminal applications. In one or more embodiments, the high-computing-power electronic devices or apparatuses according to the present disclosure can be applied to cloud devices (e.g., cloud servers), while the low-power electronic devices or apparatuses can be applied to terminal devices and / or edge devices (e.g., smartphones or cameras). In one or more embodiments, the hardware information of the cloud devices and the hardware information of the terminal devices and / or edge devices are compatible with each other, so that suitable hardware resources can be matched from the hardware resources of the cloud devices to simulate the hardware resources of the terminal devices and / or edge devices based on the hardware information of the terminal devices and / or edge devices, so as to complete the unified management, scheduling and collaborative work of end-to-cloud or cloud-edge-end integration.

[0139] It should be noted that, for the sake of brevity, this disclosure describes some methods and their embodiments as a series of actions and combinations thereof. However, those skilled in the art will understand that the solutions disclosed herein are not limited by the order of the described actions. Therefore, based on the disclosure or teachings of this document, those skilled in the art will understand that some steps can be performed in a different order or simultaneously. Furthermore, those skilled in the art will understand that the embodiments described in this disclosure can be considered optional embodiments, that is, the actions or modules involved are not necessarily essential for the implementation of one or more solutions disclosed herein. In addition, depending on the solution, the description of some embodiments in this disclosure may have different emphases. In view of this, those skilled in the art will understand that parts not described in detail in a certain embodiment of this disclosure can also be referred to the relevant descriptions of other embodiments.

[0140] In terms of specific implementation, based on the disclosure and teachings of this document, those skilled in the art will understand that several embodiments disclosed herein can also be implemented in other ways not disclosed herein. For example, regarding the various units in the electronic device or apparatus embodiments described above, this document divides them based on logical functions, but in actual implementation, there may be other division methods. As another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. Regarding the connection relationships between different units or components, the connections discussed above in conjunction with the accompanying drawings can be direct or indirect couplings between units or components. In some scenarios, the aforementioned direct or indirect couplings involve communication connections utilizing interfaces, where the communication interface can support electrical, optical, acoustic, magnetic, or other forms of signal transmission.

[0141] In this disclosure, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed across multiple network units. Furthermore, depending on actual needs, some or all of the units can be selected to achieve the purpose of the solution described in the embodiments of this disclosure. Additionally, in some scenarios, multiple units in the embodiments of this disclosure may be integrated into one unit or each unit may exist physically independently.

[0142] In some implementation scenarios, the integrated unit described above can be implemented as a software program module. If implemented as a software program module and sold or used as an independent product, the integrated unit can be stored in a computer-readable storage device (CMSDD). Therefore, when the disclosed solution is embodied in a software product (e.g., a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to cause a computer device (e.g., a personal computer, server, or network device) to execute some or all of the steps of the method described in the embodiments of this disclosure. The aforementioned memory may include, but is not limited to, various media capable of storing program code, such as USB flash drives, flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0143] In other implementation scenarios, the integrated units described above can also be implemented in hardware, i.e., as specific hardware circuits, which may include digital circuits and / or analog circuits. The physical implementation of the circuit's hardware structure may include, but is not limited to, physical devices, which may include, but are not limited to, transistors or memristors. Therefore, the various devices described herein (e.g., artificial intelligence processor computing devices or other processing devices) can be implemented using appropriate hardware processors, such as CPUs, GPUs, FPGAs, DSPs, and ASICs. Furthermore, the aforementioned storage units or storage devices can be any suitable storage medium (including magnetic storage media or magneto-optical storage media), such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), ROM, and RAM.

[0144] While numerous embodiments of this disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and intent of this disclosure. It should be understood that various alternatives to the embodiments of this disclosure described herein may be employed in the practice of this disclosure. The appended claims are intended to define the scope of this disclosure and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. An instruction processing apparatus, comprising: Decoding unit, configured to decode instructions; The microinstruction generation unit is configured to split the decoded instruction into several microinstructions. A microinstruction-level dependency determination unit is configured to determine whether there is a microinstruction-level dependency between the currently processed microinstruction and the preceding instruction being executed. as well as The instruction issuing unit is configured to determine whether to issue the currently processed microinstruction based on the microinstruction-level dependency relationship.

2. The instruction processing apparatus according to claim 1, wherein, The microinstruction-level dependency determination unit is further configured to: For any microinstruction involving a write operation, if the data memory access address of the currently processed microinstruction does not overlap with the data memory access addresses of all incomplete microinstructions in the preceding instructions, it is determined that there is no microinstruction-level dependency between the currently processed microinstruction and the preceding instructions.

3. The instruction processing apparatus according to claim 2, further comprising: An instruction-level dependency determination unit is configured to determine whether there is an instruction-level dependency between the currently processed instruction and the preceding instruction being executed. and The microinstruction-level dependency determination unit is further configured to: when the instruction-level dependency determination unit determines that an instruction-level dependency exists, further determine whether a microinstruction-level dependency exists; The instruction issuing unit is further configured to: issue the currently processed microinstruction when there is no dependency relationship between any indication in the instruction-level dependency relationship and the microinstruction-level dependency relationship.

4. The instruction processing apparatus according to claim 3, further comprising: The address range maintenance unit is configured to maintain a dynamic address range for each preceding instruction in execution. The address range includes the start and end addresses of the data memory access addresses corresponding to the incomplete portion of the preceding instruction.

5. The instruction processing apparatus according to claim 4, wherein, The address range maintenance unit is further configured to: In response to the submission of a microinstruction, the address range of its corresponding preceding instruction is updated.

6. The instruction processing apparatus according to any one of claims 4-5, wherein the instruction-level dependency determination unit is further configured to: For any instruction involving a write operation, if the starting address of the data memory access address of the currently processed instruction is greater than the current ending address of the preceding instruction being executed, it is determined that there is no instruction-level dependency; otherwise, it is determined that there is an instruction-level dependency.

7. The instruction processing apparatus according to claim 6, wherein the microinstruction-level dependency determination unit is further configured to: For microinstructions with instruction-level dependencies, if the end address in the data memory access address of the microinstruction is less than the current start address of the preceding instruction being executed, it is determined that there is no microinstruction-level dependency; otherwise, it is determined that there is a microinstruction-level dependency.

8. The instruction processing apparatus according to claim 7, wherein the microinstruction-level dependency determination unit is further configured to: For microinstructions with instruction-level dependencies, if the end address in the data memory access address of the microinstruction is less than the minimum value of limit of the current start address of all preceding instructions that have an instruction-level dependency with it, it is determined that there is no microinstruction-level dependency.

9. The instruction processing apparatus according to claim 8, wherein, The address range maintenance unit is further configured to: perform pairwise comparisons of the current starting addresses of all preceding instructions being executed to obtain a comparison matrix; and The microinstruction-level dependency determination unit is further configured to: based on the comparison matrix, and using the instruction-level dependency relationship of the currently processed microinstruction, determine the minimum value of the current starting address of all preceding instructions that have an instruction-level dependency relationship with the currently processed microinstruction.

10. The instruction processing apparatus according to any one of claims 1-8, wherein, When the operand of the current instruction is a multidimensional data block, and the operand of the preceding instruction with an instruction-level dependency relationship is also the same multidimensional data block, the microinstruction-level dependency determination unit is further configured to: The microinstruction-level dependency relationship is determined based on the dimension that the current instruction and the preceding instruction have the same access order.

11. The instruction processing apparatus according to claim 10, wherein, The microinstruction generation unit is further configured to: adjust the splitting strategy of the current instruction so that there is at least one dimension with consistent access order between the split microinstruction and the preceding instruction.

12. A processor comprising an instruction processing apparatus according to any one of claims 1-11.

13. A chip comprising the processor according to claim 12.

14. A circuit board comprising the chip according to claim 13.

15. An instruction processing method, comprising: Decode the instructions; The decoded instructions are broken down into several micro-instructions; Determine whether there is a microinstruction-level dependency between the currently processed microinstruction and the preceding instruction being executed; as well as Based on the microinstruction-level dependencies, determine whether to issue the currently processed microinstruction.

16. The instruction processing method according to claim 15, wherein, Determining whether a microinstruction-level dependency exists between the currently processed microinstruction and its preceding instruction includes: For any microinstruction involving a write operation, if the data memory access address of the currently processed microinstruction does not overlap with the data memory access addresses of all incomplete microinstructions in the preceding instructions, it is determined that there is no microinstruction-level dependency between the currently processed microinstruction and the preceding instructions.

17. The instruction processing method according to claim 16, further comprising: Determine whether there is an instruction-level dependency between the currently processed instruction and the preceding instruction being executed; Once an instruction-level dependency is determined, it is further determined whether a micro-instruction-level dependency exists. as well as When there is no dependency relationship between any of the instruction-level dependencies and the microinstruction-level dependencies, the currently processed microinstruction is emitted.

18. The instruction processing method according to claim 17, further comprising: A dynamic address range is maintained for each preceding instruction in execution, the address range including the start and end addresses of the data memory access addresses corresponding to the incomplete part of the preceding instruction.

19. The instruction processing method according to claim 18, wherein maintaining the address range includes: In response to the submission of a microinstruction, the address range of its corresponding preceding instruction is updated.

20. The instruction processing method according to any one of claims 17-18, wherein determining whether there is an instruction-level dependency between the currently processed instruction and the preceding instruction being executed includes: For any instruction involving a write operation, if the starting address of the data memory access address of the currently processed instruction is greater than the current ending address of the preceding instruction being executed, it is determined that there is no instruction-level dependency. Otherwise, an instruction-level dependency is confirmed.

21. The instruction processing method according to claim 20, wherein determining whether the microinstruction-level dependency exists includes: For microinstructions with instruction-level dependencies, if the end address in the data memory access address of the microinstruction is less than the current start address of the preceding instruction being executed, it is determined that there is no microinstruction-level dependency. Otherwise, a microinstruction-level dependency is confirmed.

22. The instruction processing method according to claim 21, wherein determining whether the microinstruction-level dependency exists further comprises: For microinstructions with instruction-level dependencies, if the end address in the data memory access address of the microinstruction is less than the minimum value of limit of the current start address of all preceding instructions that have an instruction-level dependency with it, it is determined that there is no microinstruction-level dependency.

23. The instruction processing method according to claim 22, further comprising: The current starting addresses of all preceding instructions are compared pairwise to obtain a comparison matrix; as well as Based on the comparison matrix, the minimum current starting address of all preceding instructions that have an instruction-level dependency relationship with the currently processed microinstruction is determined by utilizing the instruction-level dependency relationship of the currently processed microinstruction.

24. The instruction processing method according to any one of claims 15-23, wherein, When the operand of the current instruction is a multidimensional data block, and the operand of the preceding instruction that has an instruction-level dependency with it is also the same multidimensional data block, determining whether there is a microinstruction-level dependency between the currently processed microinstruction and the executing preceding instruction includes: The microinstruction-level dependency relationship is determined based on the dimension that the current instruction and the preceding instruction have the same access order.

25. The instruction processing method according to claim 24, further comprising: Adjust the splitting strategy of the current instruction so that there is at least one dimension in which the split microinstruction and the preceding instruction have the same access order.

Images