Command processor, related device and method

Abstract

The present disclosure provides a command processor, related devices and methods. The command processor is used to assign an instruction queue to an instruction execution unit for execution. The instruction queue is pre-segmented and stored in multiple storage areas outside the command processor. The command processor includes: a first buffer unit, used to store a first instruction stream retrieved from a first storage area, the first instruction stream includes an instruction segment of the instruction queue and pointers to other instruction segment storage areas; a first pre-fetch unit, used to analyze the first instruction stream stored in the first buffer unit, and read the storage area of ​​the instruction segment pointed to by each pointer according to the order of one or more pointers in the first instruction stream to retrieve one or more second instruction streams. The present disclosure optimizes the instruction reading mechanism, so that the pre-storage space for instructions has flexibility and improves the speed of reading instructions from the storage space.

Description

Technical Field

[0001] The present disclosure relates to the field of processors and systems on chip, and in particular, to a command processor, related devices and methods. Background Art

[0002] Traditional processing units (such as CPUs, central processing units) are not efficient in computing in some specialized fields. In order to improve the computing efficiency in specialized fields, specialized acceleration units have emerged, such as general-purpose graphics processing units (GPGPUs) used in the field of image processing, neural network processing units (NPUs) used in the field of deep learning, and some parallel processing units (PPUs) developed in multiple fields. Since the acceleration unit has built-in on-chip memory, it reduces the overhead of frequent data transfer between the processing unit and the computing process.

[0003] At present, the acceleration unit performs special computing processing under the scheduling of the processing unit. The acceleration unit stores the instructions forwarded by the processing unit to the acceleration unit through a built-in static random access memory (SRAM). Specifically, the processing unit accesses the SRAM through MMIO (Memory-mapped I / O) and writes the instructions to the SRAM one dword (full name Double Word, which refers to the number of bits of a value in a computer. Since each word is 2 bytes long, a dword is 4 bytes with a total of 32 bits) at a time and configures the write pointer of the SRAM, so that the command processor (CP) in the acceleration unit can directly read the instructions stored in the SRAM according to the write pointer.

[0004] The above instruction reading scheme is very simple and intuitive in hardware design, but it still has the following disadvantages: (1) Since the CP reads the instructions stored in the SRAM according to the write pointer, an instruction stream controlled by a write pointer can only be stored in the continuous storage space in the SRAM, that is, the storage space of the instruction is limited by hardware; (2) If the continuous storage space in the SRAM can only store one instruction stream, the processing unit cannot forward multiple instruction streams to the SRAM at the same time. The SRAM can only store instruction streams one by one. Accordingly, the CP can only read multiple instruction streams by repeating the write pointer acquisition process and the data reading process according to the write pointer, that is, the instruction reading speed is limited. Summary of the invention

[0005] In view of this, the present disclosure aims to optimize the instruction reading mechanism of the command processor, so that the pre-storage space of the instructions is more flexible and the speed of reading the instructions from the storage space is improved.

[0006] According to a first aspect of the present disclosure, a command processor is provided, the command processor being used to assign an instruction queue to an instruction execution unit for execution, the instruction queue being pre-segmented and stored in a plurality of storage areas outside the command processor, the command processor comprising:

[0007] A first buffer unit, used for storing a first instruction stream retrieved from a first storage area, wherein the first instruction stream includes an instruction segment of the instruction queue and pointers to other instruction segment storage areas;

[0008] The first prefetch unit is used to analyze the first instruction stream stored in the first buffer unit, and read the storage area of ​​the instruction segment pointed to by each pointer according to the order of one or more pointers in the first instruction stream to retrieve one or more second instruction streams.

[0009] Optionally, the command processor further includes:

[0010] The second buffer unit is used to sequentially store the second instruction streams retrieved by the first prefetch unit according to a retrieval order.

[0011] Optionally, the first instruction stream includes an instruction segment and a pointer to an instruction segment storage area following the instruction segment, the second instruction stream includes an instruction segment and a pointer to an instruction segment storage area following the instruction segment, and the command processor further includes:

[0012] The second pre-fetch unit is used to analyze the second instruction stream stored in the second buffer unit, and for the pointer included in the analyzed second instruction stream, read the storage area of ​​the instruction segment pointed to by the pointer to retrieve the instruction stream.

[0013] Optionally, at least one of the instruction streams retrieved by the second prefetch unit includes an instruction segment and a pointer to a storage area for an instruction segment following the instruction segment;

[0014] The instruction streams retrieved by the second prefetch unit are all stored in the second buffer unit in a retrieval order;

[0015] The second prefetch unit is used to analyze all instruction streams stored in the second buffer unit, and read the storage area of ​​the instruction segment pointed to by each unprocessed pointer according to the storage order of each unprocessed pointer in the second buffer unit to retrieve the instruction stream.

[0016] Optionally, the first instruction stream includes a plurality of pointers pointing to other instruction segment storage areas;

[0017] After the first prefetch unit retrieves the first instruction stream based on the current pointer in the first buffer unit, once the current instruction stream analyzed by the second prefetch unit does not include a pointer, the first prefetch unit reads the instruction segment storage area pointed to by the next pointer in the first buffer unit after the second prefetch unit completes analyzing the current instruction stream.

[0018] Optionally, the multiple storage areas are located in a memory within the same device as the instruction execution unit, the first instruction stream is retrieved from the first storage area based on a direct memory access mechanism and stored in the first buffer unit, and the first prefetch unit and the second prefetch unit both read the storage area of ​​the instruction segment pointed to by the pointer based on the direct memory access mechanism.

[0019] Optionally, the first prefetch unit is further used to parse the analyzed instruction segments and distribute them to the instruction execution unit during the process of analyzing the first instruction stream stored in the first buffer unit, and the second prefetch unit is further used to parse the analyzed instruction segments and distribute them to the instruction execution unit during the process of analyzing all instruction streams stored in the second buffer unit;

[0020] Furthermore, the first buffer unit and the second buffer unit are both ring buffers.

[0021] According to a second aspect of the present disclosure, there is provided a processing unit, comprising: an instruction execution unit and any one of the command processors described above.

[0022] According to a third aspect of the present disclosure, a computing device is provided, comprising the processing unit as described above.

[0023] According to a fourth aspect of the present disclosure, a system on chip is provided, comprising:

[0024] Any of the command processors described above;

[0025] Instruction execution unit;

[0026] The on-chip bus is used to couple the instruction execution unit and the command processor.

[0027] According to a fifth aspect of the present disclosure, a method for reading an instruction queue is provided, wherein the instruction queue is pre-segmented and stored in a plurality of storage areas outside a command processor, the method comprising:

[0028] buffering a first instruction stream retrieved from a first storage area to the command processor, the first instruction stream comprising an instruction segment of the instruction queue and pointers to other instruction segment storage areas;

[0029] The first instruction stream buffered by the command processor is analyzed, and according to the order of one or more pointers in the first instruction stream, the storage area of ​​the instruction segment pointed to by each pointer is read to retrieve one or more second instruction streams.

[0030] In the disclosed embodiment, the instruction queue is pre-segmented and stored in multiple storage areas outside the command processor, and the first instruction stream retrieved from the first storage area and stored in the first buffer unit includes an instruction segment of the instruction queue and pointers to other instruction segment storage areas, that is, an instruction segment of the instruction queue and pointers to other instruction segment storage areas are retrieved simultaneously and then stored in the first buffer unit at the same time, which saves the time of specially fetching the storage addresses of other instruction segments and improves the speed of reading the instruction queue from multiple storage areas; next, the first pre-fetch unit analyzes the first instruction stream stored in the first buffer unit and reads the storage area of ​​the instruction segment pointed to by each pointer according to the order of one or more pointers in the first instruction stream. Therefore, the above-mentioned multiple storage areas do not have to be continuous, and different instruction segments can be read out based on pointers, and the pre-storage space of the instruction queue is more flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above and other objects, features and advantages of the present disclosure will become more apparent through the description of the embodiments of the present disclosure with reference to the following drawings, in which:

[0032] Figure 1 is a structural diagram of a data center to which an embodiment of the present disclosure is applied;

[0033] Figure 2 is a structural block diagram of a server in a data center according to an embodiment of the present disclosure;

[0034] Figure 3 yes Figure 2 A detailed internal structure diagram of the server shown;

[0035] Figure 4 yes Figure 3 A schematic diagram of the core structure of the acceleration unit shown;

[0036] Figure 5 The figure shows a static random access memory with a configured write pointer in the related art;

[0037] Figure 6 is a schematic diagram of storing a data queue in multiple storage areas in one embodiment of the present disclosure;

[0038] Figure 7 is an internal structure diagram of a command processor in one embodiment of the present disclosure;

[0039] Figure 8 The first buffer unit and the second buffer unit buffer in one embodiment of the present disclosure Figure 6 A schematic diagram of multiple instruction flows shown;

[0040] Fig. 9 yes Figure 6 A sequence diagram showing that a plurality of instruction streams are analyzed sequentially;

[0041] Fig.10 is a flow chart of a method for reading an instruction queue in one embodiment of the present disclosure;

[0042] Fig.11 It is a flow chart of a method for reading an instruction queue in another embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] The present disclosure is described below based on embodiments, but the present disclosure is not limited to these embodiments. In the detailed description of the present disclosure below, some specific details are described in detail. It is possible for a person skilled in the art to fully understand the present disclosure without the description of these details. In order to avoid confusing the essence of the present disclosure, well-known methods, processes, and procedures are not described in detail. In addition, the drawings are not necessarily drawn to scale.

[0044] The following terms are used in this article.

[0045] Accelerator unit: A processing unit designed to improve the data processing speed in some special-purpose fields (for example, processing images, processing various operations of deep learning models, etc.) where traditional processing units are inefficient. Accelerator units are also called artificial intelligence (AI) processing units, including central processing units (CPUs), graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), parallel processing units (PPUs), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and dedicated intelligent acceleration hardware (for example, neural network processors NPUs).

[0046] Processing unit: A unit that performs traditional processing (not for complex operations such as image processing and various deep learning models) in the server of the data center. In addition to performing these non-image processing and various deep learning models and other complex operations, the processing unit also assumes the function of scheduling the acceleration unit and assigning the tasks that the acceleration unit needs to undertake to the acceleration unit. The processing unit can take the form of a processing unit (CPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc.

[0047] Command processor: The full name of command processor in English, referred to as CP, is an internal component of the acceleration unit, and the internal components of the acceleration unit also include the acceleration unit core. The command processor is used to assign the external instruction queue to the acceleration unit core for execution. The command processor needs to be responsible for reading the instruction queue from the outside and parsing the instructions so as to finally assign the parsed instructions to the acceleration unit core. The command processor can be regarded as an interface module of the hardware and software in the acceleration unit.

[0048] Indirect buffer chain: The full name is indirect buffer chain, or IB Chain for short. In this disclosure, IBChain is a storage method used by the software to allow the CP to indirectly address and fetch instructions. Different from direct addressing, IB Chain stores the addresses of other storage areas in a certain storage area (i.e., stores pointers to other storage areas). After the CP resolves to the address, it will issue an instruction fetch request again based on the resolved address.

[0049] Instruction segment: The instruction queue is segmented and stored in multiple storage areas, each storage area stores an instruction segment of the instruction queue. Here, the instruction queue refers to a collection of multiple instructions.

[0050] Instruction stream: refers to all instructions stored in a storage area. The instructions of the storage area can be read out in the form of a stream through the starting address and the ending address. In the present disclosure, all instructions stored in some storage areas are only instruction segments, and the instruction stream is equivalent to the instruction segment; all instructions stored in other storage areas are instruction segments and pointers to other storage areas, so the instruction stream is instruction segments and pointers.

[0051] Storage area: a continuous storage space composed of multiple storage units. In the present disclosure, one storage area corresponds to one pointer. The command processor determines the storage area based on the pointer so as to read the instruction stream from the determined storage area.

[0052] Ring buffer: A data structure used to represent a fixed-size, head-to-tail connected buffer. A useful feature is that when a data element is used up, the remaining data elements do not need to move their storage locations, making it suitable for implementing first-in, first-out. The specific working mechanism is: the read pointer indicates the start position of the valid data stored in the buffer, and the write pointer indicates the end position of the valid data stored in the buffer. If the read pointer is equal to the write pointer, it means that all the valid data has been read out.

[0053] System memory access latency: After the command processor issues a request to read the system memory, it takes a certain period of time (about 200 cycles) for the system memory to return the read data. This time interval is called system memory access latency. The main reason for the latency is that in addition to the time it takes for the system memory to respond to a single request, the system memory is accessed by other hardware devices such as the processing unit and graphics card, so there is a queue problem for request responses.

[0054] Application environment of the present disclosure

[0055] The embodiment of the present disclosure proposes a command processor, the function of which is to read back the instruction queue from the outside and then assign the read instructions to the instruction execution unit for execution. The entire instruction reading scheme adopted by the command processor is relatively common in the processor. The command processor and the instruction execution unit are both internal components of the processor. The command processor can exist as a separate device, and after being installed in a device or a system on chip, it cooperates with the instruction execution unit of the device or the system on chip to realize the function of instruction processing. The command processor can be used for various hardware devices provided with a processor, for example, a data center, an IOT (Internet of Things) device for executing an image processing algorithm, an embedded device, etc. The processor can be specifically embodied as a traditional processing unit, or it can be embodied as an acceleration unit designed for the situation that the efficiency of traditional processing units in some special use fields is not high. The command processor provided by the present disclosure is independent of the specific type of the processor and the hardware on which the processor is ultimately deployed. However, for the purpose of exemplary description, the following will mainly describe the application scenario in a data center with the processor being an acceleration unit. It should be understood by those skilled in the art that the embodiment of the present disclosure can also be applied to other application scenarios.

[0056] Data Center

[0057] Data centers are a globally collaborative network of specific devices used to transmit, accelerate, display, calculate, and store data information on the Internet network infrastructure. In the future, data centers will also become an asset for corporate competition. With the widespread application of data centers, artificial intelligence and other technologies are increasingly being applied to data centers. Deep learning and physical simulation, as important technologies of artificial intelligence, have been widely used in the computational analysis of data centers.

[0058] In traditional large data centers, the network structure is usually as follows Figure 1 As shown in the figure, it is the hierarchical inter-networking model. This model includes the following parts:

[0059] Server 140 : Each server 140 is a processing and storage entity of the data center. The processing and storage of a large amount of data in the data center are completed by these servers 140 .

[0060] Access switch 130: Access switch 130 is used to connect servers 140 to the switch in the data center. One access switch 130 connects to multiple servers 140. Access switches 130 are usually located at the top of the rack, so they are also called top of rack switches, and they are physically connected to the servers.

[0061] Aggregation switch 120: Each aggregation switch 120 connects to multiple access switches 130 and provides other services, such as firewall, intrusion detection, network analysis, etc.

[0062] Core switch 110: The core switch 110 provides high-speed forwarding for packets entering and leaving the data center, and provides connectivity for the aggregation switch 120. The network of the entire data center is divided into an L3 layer routing network and an L2 layer routing network. The core switch 110 generally provides a flexible L3 layer routing network for the network of the entire data center.

[0063] Generally, the aggregation switch 120 is the demarcation point between the L2 and L3 layer routing networks. The network below the aggregation switch 120 is the L2 network, and the network above is the L3 network. Each group of aggregation switches manages a delivery point (POD), and each POD has an independent VLAN network. The server does not need to modify the IP address and default gateway when migrating within the POD, because one POD corresponds to one L2 broadcast domain.

[0064] The spanning tree protocol (STP) is usually used between the aggregation switch 120 and the access switch 130. STP makes only one aggregation layer switch 120 available for a VLAN network, and other aggregation switches 120 are used only when a failure occurs (dashed lines in the above figure). In other words, horizontal expansion cannot be achieved at the aggregation switch 120 level, because even if multiple aggregation switches 120 are added, only one is still working.

[0065] server

[0066] Since the server 140 is the real processing device in the data center, Figure 2 FIG. 1 shows an internal structure block diagram of a server 140. Figure 2 The server 140 includes a bus-connected memory 21 , a processing unit cluster 27 , and an acceleration unit cluster 28 .

[0067] The processing unit cluster 27 includes the processing unit 23. The acceleration unit cluster 28 includes the acceleration unit 24. A server 140 may be as follows: Figure 4 As shown, a plurality of acceleration units 24 are arranged in the acceleration unit cluster 28 , and a plurality of processing units 23 are arranged in the scheduling unit cluster 27 .

[0068] The acceleration unit 24 is a processing unit designed to increase the data processing speed in a special purpose field. In view of this, the server 140 provided with the acceleration unit 24 is also often used as a computing device. The acceleration unit 24 is also called an artificial intelligence (AI) processing unit, including a central processing unit (CPU), a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a parallel processing unit (PPU), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), and a dedicated intelligent acceleration hardware (for example, a neural network processor NPU). The processing unit 23 is to schedule the acceleration unit 24 and assign the sequence of instructions to be executed to each acceleration unit 24. It can take the form of a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), and other forms.

[0069] The architecture design of the traditional processing unit makes the control unit and storage unit occupy a large part of the space in the architecture, while the space occupied by the computing unit is insufficient. Therefore, it is very effective in terms of logical control, but not efficient enough in terms of large-scale parallel computing. Therefore, various special acceleration units have been developed to more effectively improve the computing speed for different functions and different fields. The acceleration unit 24 proposed in the present invention can be a processing unit dedicated to accelerating the computing speed of the deep learning model. It adopts a data-driven parallel computing architecture for processing a large number of operations (such as convolution, pooling, etc.) of each deep learning model node. Since the data and intermediate results in the large number of operations (such as convolution, pooling, etc.) of each deep learning model node are closely related in the entire computing process, they will be frequently used. With the existing processing unit architecture, since the memory capacity in the core of the processing unit is very small, it is necessary to frequently access the external core memory, resulting in inefficient processing. By using this acceleration unit 24 dedicated to accelerating the computing speed of deep learning models, since each core has on-chip memory with a storage capacity suitable for deep learning model calculations, frequent access to memory outside the core can be avoided, which can greatly improve processing efficiency and computing performance.

[0070] The acceleration unit 24 is subject to the scheduling of the processing unit 23. Figure 2 As shown, the memory 21 stores various deep learning models, including the nodes of these models and the weight tensors of the nodes. These deep learning models are used when needed. Figure 2A processing unit 23 is deployed to an acceleration unit 24. That is, the processing unit 23 can send the address of the parameters in the model (such as the weight tensor of each node) in the memory 21 to the acceleration unit 24 in the form of instructions. When the acceleration unit 24 actually uses the deep learning model for calculation, it will directly address these parameters in the memory 21 according to the address of these parameters (such as weight tensors) in the memory 21, and temporarily store them in its on-chip memory. When the acceleration unit 24 actually uses the deep learning model for calculation, the processing unit 23 will also send the input tensor of the model to the acceleration unit 24 in the form of instructions, and temporarily store them in the on-chip memory of the acceleration unit 24. In this way, the acceleration unit 24 can perform inference calculations based on these input tensors and the parameters in the model (such as weight tensors).

[0071] Internal structure of the processing unit and acceleration unit

[0072] Combine the following Figure 3 The internal structure diagram of the processing unit 23 and the acceleration unit 24 shown in the figure specifically illustrates how the processing unit 23 schedules the acceleration unit 24 to work.

[0073] like Figure 3 As shown, the processing unit 23 includes multiple processor cores 312 and a cache 311 shared by the multiple processor cores 312. Each processor core 312 includes an instruction fetch unit 3121, an instruction decoding unit 3122, an instruction issuing unit 3123, and an instruction executing unit 3124.

[0074] The instruction fetch unit 3121 is used to transfer the instruction to be executed from the memory 21 to the instruction register (which may be Figure 3 A register in the register stack 3126 shown is used to store instructions) and receives the next instruction fetch address or calculates the next instruction fetch address according to an instruction fetch algorithm. The instruction fetch algorithm includes, for example: incrementing the address or decrementing the address according to the instruction length.

[0075] After fetching the instruction, the processing unit 23 enters the instruction decoding stage, and the instruction decoding unit 3122 decodes the fetched instruction according to the predetermined instruction format to obtain the operand acquisition information required by the fetched instruction, thereby preparing for the operation of the instruction execution unit 3124. The operand acquisition information, for example, points to an immediate number, a register, or other software / hardware that can provide a source operand.

[0076] The instruction issuing unit 3123 is located between the instruction decoding unit 3122 and the instruction executing unit 3124 and is used for scheduling and controlling instructions so as to efficiently distribute each instruction to different instruction executing units 3124, thereby making parallel operation of multiple instructions possible.

[0077] After the instruction transmitting unit 3123 transmits the instruction to the instruction executing unit 3124, the instruction executing unit 3124 starts to execute the instruction. However, if the instruction executing unit 3124 determines that the instruction should be executed by the acceleration unit 24, it forwards it to the corresponding acceleration unit 24 for execution. For example, if the instruction is an instruction for neural network inference, the instruction executing unit 3124 no longer executes the instruction, but sends the instruction to the acceleration unit 24 through the bus for execution by the acceleration unit 24.

[0078] The acceleration unit 24 includes multiple cores 246 ( Figure 3 Four cores are shown, but those skilled in the art should understand that the acceleration unit 24 may also include other numbers of cores 246), a command processor 247, a direct memory access mechanism 245, and a bus channel 241.

[0079] The bus channel 241 is a channel for instructions to enter and exit the acceleration unit 24 from the bus, wherein the instructions coming out of the acceleration unit 24 are, for example, write instructions for the processing results of the acceleration unit 24, and the instruction execution unit 3124 receives the write instructions and writes the processing results of the acceleration unit 24 into the cache 311 or the register stack 3126.

[0080] The direct memory access (DMA) mechanism 245 is a function provided by some computer bus architectures, which enables data to be written directly from an attached device to the memory of the computer motherboard. Compared with the method in which all data transmissions between devices must pass through the processing unit 23, this method greatly improves the efficiency of data access. It is precisely because of this mechanism that the core of the acceleration unit 24 can directly access the memory 21 and read the parameters in the neural network model (such as the weights of each node), etc., which greatly improves the efficiency of data access.

[0081] The command processor 247 assigns the instructions sent by the processing unit 23 to the acceleration unit 24 or the instructions read from the memory 21 through the direct memory access mechanism 245 to the core 246 for execution. Specifically, after the instruction sequence to be executed enters from the bus channel 241, it is cached in the command processor 247, and the command processor 247 selects the core 246 and assigns the instruction sequence to it for execution. In addition, the command processor 247 is also responsible for the synchronization operation between the cores 246. After the core 246 obtains the processing result, the command processor 247 also sends the processing result write instruction to the instruction execution unit 3124 of the processing unit 23 for execution. The command processor 247 is the main improved part of the embodiment of the present disclosure, and its implementation details will be described in detail later.

[0082] Acceleration unit core

[0083] Figure 4FIG. 2 is a diagram showing the internal structure of the acceleration unit core 246 according to an embodiment of the present disclosure. Figure 4 In one embodiment, the acceleration unit core 246 includes a tensor engine 610 , a pooling engine 620 , a memory copy engine 630 , a sequencer 650 , an instruction cache 640 , an on-chip memory 660 , and a constant buffer 670 .

[0084] The instruction sequence assigned by the command processor 247 to the acceleration unit core 246 first enters the instruction cache 640 cache. Then, the sequencer 650 fetches instructions from the instruction cache 640 in a first-in-first-out order, and assigns them to the tensor engine 610, the pooling engine 620, or the memory copy engine 630 for execution according to the nature of the instructions. The tensor engine 610 is responsible for processing related operations such as convolution and matrix multiplication in the neural network model. The pooling engine 620 is responsible for processing the pooling operation in the neural network model. The memory copy engine 630 is responsible for copying the operands stored in the on-chip memory 660 in the core 246 to the memory shared between the cores 246, or the on-chip memory 660 in other cores 246. The sequencer 650 decides whether to assign the instruction to the tensor engine 610, the pooling engine 620, or the memory copy engine 630 according to the nature of the operation such as convolution, matrix multiplication, pooling, or operand copy.

[0085] The on-chip memory 660 is an in-core memory for storing weight parameters in the neural network model, as well as input parameters and various intermediate results when the neural network model is actually used. The constant buffer 670 is a buffer for storing other constant parameters (e.g., hyperparameters in the neural network model) in addition to weight parameters in the neural network model. As described above, in the process of the processing unit 23 pre-configuring the neural network model in the acceleration unit 24, the processing unit 23 sends the address of the parameters in the model in the memory 21 to the acceleration unit 24 in the form of instructions. These parameters include the weights of the nodes and other parameters (e.g., hyperparameters). For the weights, the acceleration unit 24 takes it out from the corresponding position of the memory 21 and puts it in the on-chip memory 660 when the actual neural network model is operated. For other parameters, the acceleration unit 24 takes it out from the corresponding position of the memory 21 and puts it in the constant buffer 670 when the actual neural network model is operated. In addition, when the instruction to actually start inference is assigned to the core 236 for execution by the command processor 247, the input parameters in the instruction (input to the neural network model) are also stored in the on-chip memory 660. In addition, after the tensor engine 610 and the pooling engine 620 perform convolution or pooling operations, various intermediate results obtained are also stored in the on-chip memory 660 .

[0086] Command Processor

[0087] In the related art, the acceleration unit 24 has a built-in storage device for storing instruction queues loaded from the outside. An example of a storage device may be a volatile storage device such as a static random access memory (SRAM). The command processor 247 reads the instruction queue according to the write pointer configured in the SRAM. Figure 5 FIG. 2 shows an SRAM configured with a write pointer r_addr, wherein the instruction queue is written upward from the storage unit at the bottom of the SRAM. The position pointed to by the write pointer r_addr is the last storage unit in the SRAM where the instruction queue is written. Therefore, the command processor 247 reads the storage area between the bottom of the SRAM and the storage unit pointed to by the write pointer r_addr, and the instruction queue can be read out from the SRAM ( Figure 5 As shown in the figure, a long box represents a storage unit, and a storage unit stores a binary data). In this scheme, the instruction queue read out based on a write pointer r_addr can only be pre-stored in a continuous storage area in the SRAM, so the pre-storage space of the instruction queue is limited by the hardware of the SRAM. If there is only one continuous storage area of ​​2M in the SRAM, and the externally loaded instruction queue includes three sequentially arranged instruction streams such as 1M, 2M and 1M, the instruction streams can only be loaded one by one from the outside, that is, the SRAM can only store 1M instruction streams first, and the first stored 1M instruction stream can only be stored after the command processor 247 reads out the 2M instruction stream, and then the remaining 1M instruction stream can only be stored after the stored 2M instruction stream is read out by the command processor 247, and the command processor 247 can only read the three instruction streams multiple times by repeating the write pointer acquisition process and the data reading process according to the write pointer, that is, the reading speed of the entire instruction queue is slow. In view of this, the present disclosure improves the command processor 247 to adjust the scheme for the command processor 247 to read the instruction queue, wherein the instruction queue is pre-stored in a storage space outside the command processor 247 and the storage method of the instruction queue in the storage space is also improved accordingly based on the indirect buffer chain.

[0088] Specifically, the instruction queue is pre-segmented and stored in a plurality of storage areas outside the command processor 247. Examples of the storage areas are Figure 3The storage area included in the memory 21. Specifically, the memory 21 is implemented by a non-volatile storage device, and the memory 21 includes the above-mentioned multiple storage areas and each storage area in the multiple storage areas is used to store a corresponding instruction segment of the instruction queue. It should be pointed out that the instruction stream stored in the first storage area is an instruction segment of the instruction queue and a pointer to other instruction segment storage areas (hereinafter also referred to as the instruction stream stored in the first storage area as the first instruction stream). It should be emphasized that the instruction queue segmentation is to obtain multiple instruction segments after disconnecting the instruction queue multiple times in order from front to back; and the position of the pointer stored in the first storage area in the first instruction stream is consistent with the position of the instruction segment in the storage area pointed to by the pointer in the instruction queue.

[0089] For example, refer to Figure 6 , an instruction queue composed of instructions a1-a6, b1-b7, c1-c4, d1-d4, etc. is pre-divided into more than four instruction segments and stored in more than four storage areas, that is, instruction segments a1-a6 are stored in storage area A, instruction segments b1-b7 are stored in storage area B, instruction segments c1-c4 are stored in storage area C, instruction segments d1-d4 are stored in storage area D, ...; and, storage area A is used as a first storage area to store the first instruction segment a1-a6 of the instruction queue and pointers ad1, ad2 and ad3 pointing to other storage areas, the instruction segment in the storage area pointed to by pointer ad1 is after the first instruction segment, the instruction segment in the storage area pointed to by pointer ad2 is after the instruction segment in the storage area pointed to by pointer ad1, and the instruction segment in the storage area pointed to by pointer ad3 is after the instruction segment in the storage area pointed to by pointer ad2.

[0090] It should be noted that Figure 6 A row of long strip patterns formed by arranging a plurality of long strip patterns is used to represent a storage area. A storage area is composed of a plurality of continuous storage units, but different storage areas are not continuous, that is, there are one or more storage units between different storage areas that cannot be used to store instruction queues (these storage units are either damaged due to repeated use and thus cannot store instruction queues; or because they are used as status registers to store status parameters of each storage unit, they can no longer be used to store instruction queues). Figure 6 In the figure, the solid filled bars represent the storage of pointers, and the hollow filled bars represent the storage of instructions. This is only for illustration and does not represent the actual storage status.

[0091] Reference Figure 7In view of the storage method of the instruction queue in multiple storage areas, the command processor 247 sets a first buffer unit 710 and a first pre-fetch unit 720 connected to each other, wherein the first buffer unit 710 is used to store the first instruction stream retrieved from the first storage area. Here, due to an instruction segment of the instruction queue and a pointer X pointing to other instruction segment storage areas i The instruction segments are stored together as the first instruction stream in the first storage area, so that when the instruction segments are retrieved from the first storage area, the pointers X pointing to other instruction segment storage areas can be retrieved at the same time. i , thereby saving the time of specially fetching the storage addresses of other instruction segments and improving the speed of reading the instruction queue from multiple storage areas; Next, the first pre-fetch unit 720 is used to analyze the first instruction stream stored in the first buffer unit 710, and according to one or more pointers X in the first instruction stream, i Read each pointer X in order i The instruction segment storage area pointed to by the second instruction stream X is used to retrieve one or more second instruction streams. i (a pointer X i Corresponding to the retrieval of a second instruction stream X i ), so as long as the address information of the first storage area address is obtained, the instruction segment storage area and each pointer X in the first instruction stream are i The instruction segment storage area pointed to is not continuous, and multiple instruction segments can also be read out, that is, the pre-storage space of the instruction queue is more flexible. i Represents the i-th pointer in the first instruction stream, i=1,2,3,…,n, and n is the number of pointers in the first instruction stream.

[0092] Further, refer to Figure 7 The command processor 247 further includes a second buffer unit 730. The second buffer unit 730 is connected to the first pre-fetch unit 720 and is used to sequentially store the second instruction streams X fetched by the first pre-fetch unit 720 according to the fetch order. i .

[0093] Furthermore, the first instruction stream includes an instruction segment and a pointer to a storage area of ​​an instruction segment after the instruction segment, and the second instruction stream X i Includes an instruction segment X i And point to the instruction segment X i A pointer to the next instruction segment storage area; and, referring to Figure 7 The command processor 247 further includes: a second pre-fetch unit 740 for analyzing the second instruction stream X stored in the second buffer unit 730 i , for the analyzed second instruction stream X i Pointer Y included in j , read pointer Y j Point to instruction segment Yj storage area to retrieve instruction stream Y j Among them, since the second instruction stream X i Includes an instruction segment X i And point to the instruction segment X i The pointer to the storage area of ​​the next instruction segment, so instruction segment X i and instruction segment X i The address of the next instruction segment is closely connected in terms of reading time, thereby also improving the speed of reading instruction queues from multiple storage areas. j represents the pointer read by the second prefetch unit 740 at the jth time, j=1, 2, 3, ..., m, and m is the total number of pointers read by the second prefetch unit 740.

[0094] For example, refer to Figure 6 , the instruction stream in the storage area B is the second instruction stream X1. The second instruction stream X1 is as follows Figure 8 The second pre-fetch unit 740 analyzes the second instruction stream X1 stored in the second buffer unit 730, and for the analyzed pointer Y1 (i.e. Figure 8 The pointer bd1 in the read pointer Y1 points to the storage area of ​​the instruction segment Y1 (i.e., the storage area C) to retrieve the instruction stream Y j (i.e. the instruction stream within storage area C).

[0095] Further, the instruction stream Y retrieved by the second prefetch unit 740 j At least one includes an instruction segment and a pointer to the storage area of ​​the next instruction segment after the instruction segment; the instruction stream retrieved by the second pre-fetch unit 740 is stored in the second buffer unit 730 in the order of retrieval; the second pre-fetch unit 740 is used to analyze all instruction streams stored in the second buffer unit 730, and read the storage area of ​​the instruction segment pointed to by each unprocessed pointer according to the storage order of each unprocessed pointer in the second buffer unit 730 to retrieve the instruction stream. The unprocessed pointer mentioned here refers to the pointer of the instruction segment pointed to by it that has not been retrieved. In this way, the instruction segment X i Subsequent multiple instruction segments Y j The address retrieval and the corresponding previous instruction segment are closely connected in time, thereby realizing an efficient reading process of the instruction segments one after another, and effectively improving the speed of reading the instruction queue from multiple storage areas.

[0096] For example, refer to Figure 6 , the instruction stream in the storage area C is the instruction stream Y1, and the second pre-fetch unit 740 analyzes the instruction stream Y1 stored in the second buffer unit 730 to analyze the pointer Y2 (i.e. Figure 6The pointer Y2 points to the storage area D, so the second pre-fetch unit 740 reads the storage area D to retrieve the instruction stream stored in the storage area D (i.e., the instruction stream Y2). If the instruction stream Y2 includes a pointer, the second pre-fetch unit 740 further processes the pointer. If the instruction stream Y2 is as follows: Figure 6 If no pointer is included, the second buffer unit 730 does not include any unprocessed pointer, thus terminating the entire instruction fetch process originating from a pointer in the first instruction stream.

[0097] In the above example, the second pre-fetch unit 740 analyzes all instruction streams stored in the second buffer unit 730, including analyzing the second instruction stream X i and the instruction stream Y retrieved by the second prefetch unit 740 itself j That is, the instruction stream retrieved by the first prefetch unit 720 and the instruction stream retrieved by the second prefetch unit 740 are both executed by reading the second buffer unit 730. This process sequentially analyzes multiple instruction streams in the same storage space, which is beneficial to improving the analysis speed of the second prefetch unit 740 for the retrieved instruction stream.

[0098] It should be emphasized that the order of pointers and instruction segments in the same instruction stream is related to the order of the instruction segment pointed to by the pointer and the instruction segment included in the instruction stream in the instruction queue. In view of this, the first instruction stream sets a pointer to the storage area of ​​the instruction segment after the included instruction segment; the second instruction stream X i A pointer to a storage area for an instruction segment after the instruction segment is stored at a position after the included instruction segment; and the instruction stream Y retrieved by the second pre-fetch unit 740 j If an instruction segment and a pointer to an instruction segment storage area following the instruction segment are included, a pointer to the instruction segment storage area following the instruction segment is set at a position after the included instruction segment.

[0099] It should be noted that the first pre-fetch unit 720 analyzes the instruction stream in the first buffer unit 710, and the second pre-fetch unit 740 analyzes the instruction stream in the second buffer unit 730, all in sequence according to the order of instructions and pointers in the instruction stream. In view of this, if the first instruction stream includes an instruction segment and multiple pointers pointing to all other instruction segments, the first pre-fetch unit 720 needs to perform the following jump process multiple times: after analyzing a pointer in the first buffer unit 710, read the instruction segment in multiple storage areas and store the retrieved instruction segment in the second buffer unit 730, and then jump from the second buffer unit 730 back to the first buffer unit 710 to analyze subsequent instructions or pointers. Since each jump requires recording the unanalyzed part of the first instruction stream, it is relatively wasteful of resources. The above-mentioned second instruction stream X i Includes an instruction segment X iAnd point to the instruction segment X i The pointer of the next instruction segment storage area, and even the instruction stream Y retrieved by the second pre-fetch unit 740 j At least one of the buffers includes an instruction segment and a pointer to a storage area of ​​an instruction segment following the instruction segment, which reduces the number of jumps back and forth between the first buffer unit 710 and the second buffer unit 730, thereby helping to reduce waste of resources.

[0100] for Figure 6 The entire instruction queue shown in FIG. 1 is a first buffer unit 710. Figure 8 As shown, the first instruction stream (i.e., instruction segments a1-a6 and pointers ad1, ad2, and ad3) is stored; since the first prefetch unit 720 first analyzes the pointer ad1, and then the second prefetch unit 740 analyzes the pointers bd1 and cd1 in sequence, and then the first prefetch unit 720 continues to analyze the pointer ad2 and the subsequent pointer ad3, the second buffer unit 730 should analyze the pointer ad2 in sequence as shown in FIG. Figure 8 The illustrated storage instruction stream X1 (ie, instruction segments b1-b7 and pointer bd1), instruction stream Y1 (ie, instruction segments c1-c4 and pointer cd1), and instruction stream Y2 (ie, instruction segments d1-d4).

[0101] To understand more clearly Figure 6 The order in which the instructions and pointers are analyzed can be found in Fig. 9 . Fig. 9 The instructions and pointers are arranged in the order of analysis, where the leftmost instruction a1 is the first instruction analyzed, and the pointers ad2 and ad3 listed on the right are followed by instructions and even pointers read based on them, such as pointer ad1. Here, the instructions and even pointers read based on pointers ad2 and ad3 are represented by long bars marked with ellipsis.

[0102] against Figure 6 In the case where the first instruction stream shown in the figure includes multiple pointers, the first prefetch unit 720 prefetches the first instruction stream based on the current pointer X in the first buffer unit 710. i Retrieve a second instruction stream X i After that, once the current instruction stream analyzed by the second prefetch unit 740 does not include a pointer, the first prefetch unit 720 reads the next pointer X in the first buffer unit 710 after the second prefetch unit 740 finishes analyzing the current instruction stream. i+1 The instruction segment storage area pointed to thereby realizes flexible jump of the instruction processing process between the first buffer unit 710 and the second buffer unit 730 for the case where the first instruction stream includes multiple pointers.

[0103] In some embodiments, the first prefetch unit 720 and the second prefetch unit 740 are also responsible for parsing instructions, that is, the first prefetch unit 720 is also used to parse the analyzed instruction segments and distribute them to the acceleration unit core 246 (i.e., instruction execution unit) during the process of analyzing the instruction stream in the first buffer unit 710 so that the acceleration unit core 246 can execute them, and the second prefetch unit 740 is also used to parse the analyzed instruction segments and distribute them to the acceleration unit core 246 so that the acceleration unit core 246 can execute them during the process of analyzing the instruction stream in the second buffer unit 730; and Figure 8 As shown, the first buffer unit 710 and the second buffer unit 730 are both ring buffers.

[0104] Specifically, the first buffer unit 710 and the second buffer unit 730 can each be a volatile storage device such as a static random access memory in hardware, and then be constructed into a virtual ring buffer by software. The virtual ring buffer can achieve similar functions to the ring buffer by configuring a read pointer and a write pointer. Figure 8 In the first buffer unit 710 shown, in this way, the write pointer wptr is in front and the read pointer rptr is in the back, and the buffer area between the read pointer rptr and the write pointer wptr is the area to be analyzed, and if the write pointer wptr makes a circle and writes to the buffer unit occupied by the analyzed instruction / pointer, the analyzed instruction / pointer can be overwritten and the instruction / pointer read from the above multiple storage areas can be written into the buffer unit. Since the analyzed instructions in the first buffer unit 710 and the second buffer unit 730 are allocated to the acceleration unit core 246 and the analyzed pointers have been used to retrieve the instruction stream, overwriting the analyzed instruction / pointer here does not cause adverse effects, and also saves the storage space configured for the first buffer unit 710 and the second buffer unit 730.

[0105] The plurality of storage areas are located in the memory 21 of the same server 140 as the acceleration unit 24, and the instruction stream stored in the first buffer unit 710 is based on Figure 3 The direct memory access mechanism 245 shown is retrieved from the first storage area, and the first pre-fetch unit 720 and the second pre-fetch unit 740 both read the storage area in the memory 21 based on the direct memory access mechanism 245. Compared with the related art in which the instructions are written into the command processor 247 by the processing unit 23 based on MMIO, the reading of the instructions here is no longer limited by the execution speed of the processing unit 23, and the above-mentioned solution in which the instruction segment and the pointer are retrieved in one instruction stream at the same time also further improves the speed of the direct memory access mechanism 245 to read the instructions, thereby compensating for the system memory access delay.

[0106] The first pre-fetch unit 720 and the second pre-fetch unit 740 read the instruction stream from the memory 21 and are initiated by the analyzed pointer, and the retrieval process of the instruction stream stored in the first buffer unit 710 can be initiated by configuring the virtual ring buffer by software, as follows: Figure 8 A configuration information storage area 712 is opened in the first buffer unit 710 shown (the first buffer unit 710 also includes a free area 711 that does not store any data, and the second buffer unit 730 also includes an own area 731 that does not store any data) to store the write pointer wptr' and the read pointer rptr' configured for the first storage area. When the write pointer wptr' and the read pointer rptr' are the same, the first storage area is not read. If there is an instruction stream to be read in the first storage area, the write pointer wptr' is updated so that the instruction stream between the read pointer rptr' and the write pointer wptr' is the instruction stream to be read. At the same time, the command processor 247 can have a built-in startup unit to determine whether the write pointer wptr' and the read pointer rptr' of the first storage area are the same, and if they are not the same, send a read instruction to the direct memory access mechanism 245 so that the direct memory access mechanism 245 can retrieve the first instruction stream from the first storage area.

[0107] How to read the command queue

[0108] Fig.10 is a flow chart of a method for reading an instruction queue provided by an embodiment of the present disclosure, wherein the instruction queue is pre-segmented and stored in multiple storage areas outside the command processor 247. Fig.10 , the method in one embodiment comprises:

[0109] Step S110, buffering the first instruction stream retrieved from the first storage area to the command processor 247, the first instruction stream including an instruction segment of the instruction queue and pointers to other instruction segment storage areas;

[0110] Step S120, analyzing the first instruction stream buffered by the command processor 247, and reading the storage area of ​​the instruction segment pointed to by each pointer according to the order of one or more pointers in the first instruction stream to retrieve one or more second instruction streams.

[0111] The method of the disclosed embodiment is executed by any one of the command processors 247 described above. The command processor 247 first retrieves the first instruction stream from the first storage area. Since the first instruction stream includes an instruction segment of the instruction queue and pointers to other instruction segment storage areas at the same time, the time for specially obtaining the storage addresses of other instruction segments is saved, and the speed of reading the instruction queue from multiple storage areas is improved; then, the storage area pointed to by each pointer is read according to the order of one or more pointers in the first instruction stream. Therefore, the above-mentioned multiple storage areas do not have to be continuous, and the instruction queues can all be read out based on the pointers, and the pre-storage space of the instruction queue is more flexible.

[0112] It should be understood that the main improvement of the present disclosure is to improve the command processor 247 to adjust the scheme of the command processor 247 reading the instruction queue, wherein the instruction queue is pre-segmented based on the indirect buffer chain. As for the specific use of the indirect buffer chain, there are multiple schemes to choose from. Figure 6 The solution shown is only one of them. Figure 6 The scheme shown provides a method for reading the corresponding instruction queue.

[0113] against Figure 6 The instruction queue is pre-stored in multiple storage areas (i.e., the first instruction segment is stored in storage area A and multiple pointers are stored after the instruction segment, the second instruction segment is stored in storage area B and a pointer to storage area C is stored after the instruction segment, the third instruction segment is stored in storage area C and a pointer to storage area D is stored after the instruction segment, and only the fourth instruction segment is stored in storage area D...), the detailed execution process of step S120 can be as follows Fig.11 As shown, that is, including:

[0114] Step S121, analyzing the unanalyzed portion of the first instruction stream stored in the first buffer unit 710 and determining whether the pointer X is analyzed. i , wherein, if no analysis is performed, step S122 is executed, otherwise, step S123 is executed;

[0115] Step S122, parsing the instruction and assigning the parsed instruction to the acceleration unit core 426, and then returning to execute step S121;

[0116] Step S123, read pointer X i Point to the storage area of ​​the instruction segment to retrieve the second instruction stream X i , and will fetch the second instruction stream X i Buffer to the second buffer unit 730 of the command processor, and then execute step S124;

[0117] Step S124, analyzing the unanalyzed portion of the instruction stream stored in the second buffer unit 730 and determining whether the pointer Y is analyzed. j , wherein, if no analysis is performed, step S125 is executed, otherwise, step S127 is executed;

[0118] Step S125, parsing the instruction and assigning the parsed instruction to the acceleration unit core, and then executing step S126;

[0119] Step S126, determining whether the instruction stream stored in the second buffer unit 730 has been analyzed, wherein if the analysis has not been completed, executing step S124, otherwise executing step S128;

[0120] Step S127, read pointer Y j Point to the storage area of ​​the instruction segment to retrieve the instruction stream Y j , and the retrieved instruction stream Y j Buffer to the second buffer unit 730, and then execute step S124;

[0121] Step S128, determining whether the first instruction stream stored in the first buffer unit 710 has been analyzed, wherein if the analysis has not been completed, executing step S121, otherwise, the entire reading process of the instruction queue is terminated.

[0122] It should be noted that for other different pre-storage methods of the instruction queue, the above steps S121 to S128 can be flexibly adjusted based on specific circumstances. For example, if only the pointer ad1 is stored in the storage area A, step S128 does not need to be executed. Since the instruction queue reading method for other pre-storage methods can be obtained by partially adjusting steps S121 to S128 in the previous device embodiment, the corresponding reading methods of various optional solutions will not be listed one by one here.

[0123] The commercial value of the disclosed embodiments

[0124] The command processor 247 provided in the embodiment of the present disclosure can be used in acceleration units such as GPU, GPGPU, PPU and NPU and traditional processing units, so the hardware used in various fields is improved. It has been verified through experiments that the improvement of the command processor 247 not only reduces the limitation of the pre-storage space of the instruction queue, but also effectively improves the reading speed of the instruction queue, better meets the high-speed computing needs of various fields, and has a good market prospect.

[0125] It should be understood that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the method embodiment, since it is basically similar to the method described in the device and system embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of other embodiments.

[0126] It should be understood that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0127] It should be understood that an element described in singular form herein or shown in the drawings as only one does not limit the number of the element to one. In addition, modules or elements described or shown as separate herein may be combined into a single module or element, and modules or elements described or shown as single herein may be split into multiple modules or elements.

[0128] It should also be understood that the terms and expressions used herein are for description only, and one or more embodiments of this specification should not be limited to these terms and expressions. The use of these terms and expressions does not mean to exclude any equivalent features of the illustrations and descriptions (or parts thereof), and it should be recognized that various modifications that may exist should also be included in the scope of the claims. Other modifications, changes and substitutions may also exist. Accordingly, the claims should be deemed to cover all such equivalents.

Claims

1. A command processor, for allocating a command queue to an instruction execution unit for execution, wherein the command queue is pre-segmented and stored in a plurality of storage areas outside the command processor, the command processor comprising: A first buffer unit, used for storing a first instruction stream retrieved from a first storage area, wherein the first instruction stream includes an instruction segment of the instruction queue and pointers to other instruction segment storage areas; a first prefetch unit, configured to analyze the first instruction stream stored in the first buffer unit, and read the storage area of ​​the instruction segment pointed to by each pointer according to the order of one or more pointers in the first instruction stream to retrieve one or more second instruction streams; Among them, one of the storage areas corresponds to one of the pointers.

2. The command processor according to claim 1, further comprising: The second buffer unit is used to sequentially store the second instruction streams retrieved by the first prefetch unit according to a retrieval order.

3. The command processor according to claim 2, wherein: The first instruction stream includes an instruction segment and a pointer to an instruction segment storage area after the instruction segment, the second instruction stream includes an instruction segment and a pointer to an instruction segment storage area after the instruction segment, and the command processor further includes: The second pre-fetch unit is used to analyze the second instruction stream stored in the second buffer unit, and for the pointer included in the analyzed second instruction stream, read the storage area of ​​the instruction segment pointed to by the pointer to retrieve the instruction stream.

4. The command processor according to claim 3, wherein: At least one of the instruction streams retrieved by the second prefetch unit includes an instruction segment and a pointer to a storage area for an instruction segment following the instruction segment; The instruction streams retrieved by the second prefetch unit are all stored in the second buffer unit in a retrieval order; The second prefetch unit is used to analyze all instruction streams stored in the second buffer unit, and read the storage area of ​​the instruction segment pointed to by each unprocessed pointer according to the storage order of each unprocessed pointer in the second buffer unit to retrieve the instruction stream.

5. The command processor according to claim 4, wherein: The first instruction stream includes a plurality of pointers pointing to other instruction segment storage areas; After the first prefetch unit retrieves the first instruction stream based on the current pointer in the first buffer unit, once the current instruction stream analyzed by the second prefetch unit does not include a pointer, the first prefetch unit reads the instruction segment storage area pointed to by the next pointer in the first buffer unit after the second prefetch unit completes analyzing the current instruction stream.

6. The command processor according to claim 4, wherein: The multiple storage areas are located in a memory in the same device as the instruction execution unit, the first instruction stream is retrieved from the first storage area based on a direct memory access mechanism and stored in the first buffer unit, and the first prefetch unit and the second prefetch unit both read the storage area of ​​the instruction segment pointed to by the pointer based on the direct memory access mechanism.

7. The command processor according to claim 4, wherein: The first prefetch unit is further used to parse the analyzed instruction segments and distribute them to the instruction execution unit during the process of analyzing the first instruction stream stored in the first buffer unit, and the second prefetch unit is further used to parse the analyzed instruction segments and distribute them to the instruction execution unit during the process of analyzing all instruction streams stored in the second buffer unit; Furthermore, the first buffer unit and the second buffer unit are both ring buffers.

8. A processing unit, comprising: An instruction execution unit and a command processor according to any one of claims 1-7.

9. A computing device comprising the processing unit according to claim 8.

10. A system on chip, comprising: A command processor according to any one of claims 1 to 7; Instruction execution unit; The on-chip bus is used to couple the instruction execution unit and the command processor.

11. A method for reading an instruction queue, wherein the instruction queue is pre-segmented and stored in a plurality of storage areas outside a command processor, the method comprising: buffering a first instruction stream retrieved from a first storage area to the command processor, the first instruction stream comprising an instruction segment of the instruction queue and pointers to other instruction segment storage areas; Analyzing the first instruction stream buffered by the command processor, and reading the storage area of ​​the instruction segments pointed to by each pointer according to the order of one or more pointers in the first instruction stream to retrieve one or more second instruction streams; Among them, one of the storage areas corresponds to one of the pointers.

Images