Processing system, instruction transmission method, electronic device, and storage medium

Abstract

Embodiments of the present application provide a processing system, an instruction transmission method, an electronic device and a storage medium. The processing system comprises: at least one main processor configured to output processor instructions; at least one interface unit corresponding to the at least one main processor, the interface unit comprising: an interface front end connected to the main processor, configured to receive the processor instructions, generate accelerator instructions corresponding to the processor instructions, and send the accelerator instructions; an interface register configured to receive the sent accelerator instructions and forward the accelerator instructions or receive responses to the accelerator instructions; at least one accelerator configured to process the forwarded accelerator instructions to obtain responses to the accelerator instructions; and a first bus network connected between the at least one interface unit and the at least one accelerator, configured to transmit the accelerator instructions to the corresponding accelerator or transmit the responses to the accelerator instructions to the corresponding interface register. Embodiments of the present application improve the flexibility of computing power configuration.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a processing system, instruction transmission method, electronic device, and storage medium. Background Technology

[0002] An accelerator is a device designed to handle specific computationally intensive tasks. The main processor of a processing system typically offloads these computational tasks to an accelerator, allowing the main processor to continue performing other tasks. Graphics accelerators are perhaps the most well-known, as they are available in almost all current-generation personal computers. However, there are many different types of accelerators.

[0003] Traditionally, accelerators connect to and communicate with the main processor via an external bus, such as the Peripheral Component Interconnect High Speed ​​(PCIe) bus. However, more recently, accelerators and processing systems known as DSAs have been integrated onto the same chip.

[0004] For increasingly complex machine learning models and artificial intelligence learning tasks, there is still room for improvement in the computing power configuration of current integrated accelerator and processor processing systems. Summary of the Invention

[0005] In view of the above, embodiments of this application provide a processing system, an instruction transmission method, an electronic device, and a storage medium to at least partially solve the above problems.

[0006] According to a first aspect of the embodiments of this application, a processing system is provided, comprising: at least one main processor for outputting processor instructions; at least one interface unit configured corresponding to the at least one main processor, the interface unit comprising: an interface front-end connected to the main processor for receiving the processor instructions, generating accelerator instructions corresponding to the processor instructions, and sending the accelerator instructions; an interface register for receiving the sent accelerator instructions and forwarding the accelerator instructions or receiving a response to the accelerator instructions; at least one accelerator for processing the forwarded accelerator instructions and obtaining a response to the accelerator instructions; and a first bus network connected between the at least one interface unit and the at least one accelerator for transmitting the accelerator instructions to a corresponding accelerator or transmitting the response to the accelerator instructions to a corresponding interface register.

[0007] In another implementation of the present invention, the first bus network includes at least one bus front end and at least one bus back end, wherein the at least one bus front end is configured corresponding to at least one interface register, and the at least one bus back end is configured corresponding to at least one accelerator. The first bus network is used to transmit accelerator instructions to the corresponding accelerator, or to transmit responses to accelerator instructions to the corresponding interface register, through routing between the at least one bus front end and the at least one bus back end.

[0008] In another implementation of the invention, the first bus network includes a switch disposed between the at least one bus front end and the at least one bus back end for routing the accelerator command or a response to the accelerator command between the at least one bus front end and the at least one bus back end.

[0009] In another implementation of the present invention, the first bus network includes a first sub-network and a second sub-network, the at least one main processor includes a first main processor and a second main processor, and the at least one interface unit includes a first interface unit corresponding to the first processor and a second interface unit corresponding to the second main processor. The interface register of the first interface unit is connected to the at least one accelerator through the first sub-network, and the interface register of the second interface unit is connected to the at least one accelerator through the second sub-network.

[0010] In another implementation of the invention, the first bus network includes a first sub-network and a second sub-network, and the at least one accelerator includes a first accelerator and a second accelerator. The interface register is connected to the first accelerator through the first sub-network, and the interface register is connected to the second accelerator through the second sub-network.

[0011] In another implementation of the present invention, the at least one main processor includes a plurality of main processors, the at least one accelerator includes a plurality of accelerators, and the first bus network includes a plurality of sub-networks. The plurality of main processors, the plurality of accelerators, and the plurality of sub-networks are respectively corresponding, and the main processors among the plurality of main processors are connected to the corresponding accelerators through the corresponding sub-networks.

[0012] According to a second aspect of the embodiments of this application, a processing system is provided, comprising: a first main processor for outputting first processor instructions; a first interface unit configured corresponding to the first main processor, the first interface unit comprising: an interface front-end connected to the first main processor for receiving the first processor instructions, generating a transmission instruction corresponding to the processor instructions, and sending the transmission instructions; an interface register for receiving the sent transmission instructions and forwarding the transmission instructions or receiving a response to the transmission instructions; a second main processor for at least processing second processor instructions to obtain a processing result of the second processor instructions; a second interface unit configured corresponding to the second main processor, the second interface unit comprising: an interface register for receiving the sent transmission instructions and forwarding the transmission instructions or receiving a response to the transmission instructions; an interface front-end connected to the second main processor for receiving the transmission instructions, generating a second processor instruction corresponding to the transmission instructions, or generating a response to the transmission instructions based on the processing result of the second processor instructions; and a third bus network connected between the first main processor and the second main processor for transmitting the transmission instructions from the first main processor to the second main processor, or transmitting the response to the transmission instructions from the second main processor to the first main processor.

[0013] In another implementation of the present invention, the first main processor is further configured to output a second processor instruction. The interface front-end of the first interface unit is further configured to receive the second processor instruction, generate a second accelerator instruction corresponding to the second processor instruction, and send the second accelerator instruction. The interface register of the first interface unit is further configured to receive the sent second accelerator instruction and forward the second accelerator instruction or receive a response to the second accelerator instruction. The processing system further includes: at least one accelerator for processing the forwarded second accelerator instruction and obtaining a response to the second accelerator instruction; and a first bus network connected between the first interface unit and the at least one accelerator for transmitting the second accelerator instruction to the corresponding accelerator or transmitting a response to the second accelerator instruction to the interface register of the first interface unit.

[0014] In another implementation of the present invention, the second main processor is further configured to output a third processor instruction. The interface front-end of the second interface unit is further configured to receive the third processor instruction, generate a third accelerator instruction corresponding to the third processor instruction, and send the third accelerator instruction; the interface register of the second interface unit is further configured to receive the sent third accelerator instruction, and forward the third accelerator instruction or receive a response to the third accelerator instruction. The processing system further includes: at least one accelerator for processing the forwarded third accelerator instruction and obtaining a response to the third accelerator instruction; and a fourth bus network connected between the second interface unit and the at least one accelerator for transmitting the third accelerator instruction to the corresponding accelerator or transmitting the response to the third accelerator instruction to the interface register of the second interface unit.

[0015] In another implementation of the present invention, the first main processor is a scheduling processor and the second main processor is an execution processor.

[0016] According to a third aspect of the embodiments of this application, an instruction transmission method is provided, applied to an interface unit, comprising: receiving a processor instruction from a main processor connected to the interface unit; generating an accelerator instruction corresponding to the processor instruction; determining a target accelerator corresponding to the accelerator instruction from a plurality of accelerators connected to a first bus network; and sending the accelerator instruction to the target accelerator through the first bus network.

[0017] In another implementation of the present invention, determining the target accelerator corresponding to the accelerator instruction from a plurality of accelerators connected to the first bus network includes: determining the number of instructions to be processed for each of the plurality of accelerators connected to the first bus network; and determining the target accelerator among the plurality of accelerators such that the number of instructions to be processed for the target accelerator is less than the number of instructions to be processed for the other accelerators.

[0018] In another implementation of the present invention, sending the accelerator instruction to the target accelerator through the first bus network includes: sending the accelerator instruction and the identifier of the target accelerator to a switch of the first bus network through a bus front-end corresponding to the interface unit of the first bus network, wherein the identifier instructs the switch to locate the target accelerator and forward the accelerator instruction to the target accelerator.

[0019] In another implementation of the present invention, the method further includes: receiving a processing response of the accelerator instruction returned by the target accelerator through the first bus network.

[0020] According to a fourth aspect of the embodiments of this application, an electronic device is provided, comprising: a processing system, a memory, a communication interface, and a fifth bus network as described in the first or second aspect, wherein the processing system, the memory, and the communication interface communicate with each other through the fifth bus network; the memory is used to store at least one executable instruction of a computer program, and the main processor of the processing system generates processor instructions based on the at least one executable instruction.

[0021] According to a fifth aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, the program being executed according to the processing system described in the first or second aspect.

[0022] In the processing system according to the embodiments of this application, at least one interface unit is configured corresponding to at least one main processor. Instruction translation from the main processor to the accelerator is achieved through the interface front-end. The interface register is used to forward the translated instructions, which facilitates instruction queue management. Thus, instructions suitable for accelerator processing are obtained through the interface unit. Furthermore, a first bus network is connected between at least one interface unit and at least one accelerator, used to transmit accelerator instructions to the corresponding accelerator or to transmit accelerator instruction responses to the corresponding interface register. The first bus network enables a flexible number of accelerators to be bus-connected to the main processor, thereby adapting to a wider variety of machine learning models and artificial intelligence learning tasks, and improving the flexibility of computing power configuration. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0024] Figure 1 This is a schematic block diagram of a processing system according to an embodiment of the present invention.

[0025] Figure 2 This is a schematic block diagram of a processing system according to another embodiment of the present invention.

[0026] Figures 3A-3E for Figure 2 Schematic block diagrams of various variations of the processing system in the embodiments.

[0027] Figure 4 This is a schematic block diagram of a processing system according to another embodiment of the present invention.

[0028] Figures 5A-5D for Figure 4Schematic block diagrams of various variations of the processing system in the embodiments.

[0029] Figure 6 This is a flowchart of the steps of an instruction transmission method according to another embodiment of the present invention.

[0030] Figure 7 This is a schematic diagram of the structure of an electronic device according to another embodiment of the present invention. Detailed Implementation

[0031] Exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art.

[0032] Figure 1 A block diagram illustrating an example of a processing system 100 according to the present invention is shown. Figure 1 As shown, the processing system 100 includes a main processor 110, which includes a main decoder 112, a multi-word GPR 114 connected to the main decoder 112, and an input stage 116 connected to the main decoder 112 and the GPR 114. Furthermore, the main processor 110 includes an execution stage 120 connected to the input stage 116 and a switch 122 connected to the main decoder 112, the execution stage 120, and the GPR 114.

[0033] The main decoder 112, GPR 114, input stage 116, and execution stage 120 are common components found in main processors such as RISC-V processors. For example, the GPR in a RISC-V processor has 32 memory locations, each 32 bits long. Furthermore, the execution stage typically includes an arithmetic logic unit (ALU), a multiplier, and a load-memory unit (LSU).

[0034] like Figure 1 As further shown, the processing system 100 also includes an interface unit 130 connected to the input stage 116 and the switch 122 of the main processor 110. The interface unit 130 includes a front end 132 connected to the input stage 116, an interface decoder 134 connected to the front end 132, and a timeout counter 136 connected to the front end 132.

[0035] The interface unit 130 also includes multiple interface registers RG1-RGn, each connected to the front end 132 and the interface decoder 134. Each interface register RG has a command register 140 with multiple 32-bit command memory locations C1-Cx, and a response register 142 with multiple 32-bit response memory locations R1-Ry.

[0036] Although this example shows each command register 140 as having the same number of command storage locations Cx, command registers 140 may alternately have different numbers of command storage locations C. Similarly, although this example shows each response register 142 as having the same number of response storage locations Ry, response registers 142 may alternately have different numbers of response storage locations R.

[0037] Furthermore, each of the interface registers RG has a first-in-first-out (FIFO) output queue 144 connected to the command register 140 and a FIFO input queue 146 connected to the response register 142. Each row of the FIFO output queue 144 has the same number of memory locations as the command register 140. Similarly, each row of the FIFO input queue 146 has the same number of memory locations as the response register 142.

[0038] Furthermore, interface unit 130 includes an output multiplexer 150 connected to interface decoder 134 and each interface register RG. Optionally, interface unit 130 may include an out-of-index detector 152 connected to interface decoder 134. Additionally, interface unit 130 includes a switch 154 connected to front end 132, which selectively connects timeout counter 136, multiplexer 150, or out-of-index detector 152 (when used) to switch 122.

[0039] like Figure 1 As further shown, the processing system 100 also includes multiple (DSA) DSA1-DSAn connected to the output queue 144 and input queue 146 of the interface registers RG1-RGn. DSAs can be implemented using various conventional accelerators, such as those for video, vision, artificial intelligence, vector, and general matrix multiplication. Furthermore, DSAs can operate at any desired clock frequency.

[0040] As described in more detail below, many new instructions, including accelerator write, push-ready, push, read-ready, pop, and read instructions, are added to the standard instruction set architecture (ISA). For example, RISC-VISA has four basic instruction sets (RV32I, RV32E, RV64I, RV128I) and several extended instruction sets (e.g., M, A, F, D, G, Q, C, L, B, J, T, P, V, N, H) that can be added to the basic instruction sets to achieve specific goals. In this example, RISC-VISA is modified to include the new instructions in a custom extended set.

[0041] Furthermore, the new instructions use the same instruction format as other instructions in ISA. For example, RISC-VISA has six instruction formats. One of the six formats is the Type I format, which has a seven-bit opcode field, a five-bit destination field that identifies the destination location in the general purpose register (GPR), a three-bit function field that identifies the operation, a five-bit operand field that identifies the operand's position in the GPR, and a 12-bit immediate field.

[0042] Figure 2 This is a schematic block diagram of a processing system according to another embodiment of the present invention. The processing system of this embodiment includes at least one main processor 210, at least one interface unit 220, a first bus network 230, and at least one accelerator 240. As an example, the at least one interface unit 220 can be implemented as interface unit 130, and the at least one main processor 210 can be implemented as main processor 110.

[0043] At least one main processor 210 is used to output processor instructions.

[0044] At least one interface unit 220 is configured corresponding to at least one main processor. The interface unit 220 includes an interface front-end 221 (e.g., front-end 132) and an interface register 222 (e.g., interface registers RG1-RGn). For example, each interface unit includes at least one interface front-end and at least one interface register.

[0045] Specifically, the interface front end 221 is connected to the main processor. The interface front end 221 is used to receive processor instructions, generate accelerator instructions corresponding to the processor instructions, and send the accelerator instructions. The interface register 222 is used to receive the sent accelerator instructions and forward the accelerator instructions or receive the response to the accelerator instructions.

[0046] At least one accelerator 240 is used to process the forwarded accelerator commands and obtain a response to the accelerator commands;

[0047] The first bus network 230 is connected between at least one interface unit and at least one accelerator, and is used to transmit accelerator instructions to the corresponding accelerator or to transmit the response of the accelerator instruction to the corresponding interface register.

[0048] In the processing system according to the embodiments of this application, at least one interface unit is configured corresponding to at least one main processor. Instruction translation from the main processor to the accelerator is achieved through the interface front-end. The interface register is used to forward the translated instructions, which facilitates instruction queue management. Thus, instructions suitable for accelerator processing are obtained through the interface unit. Furthermore, a first bus network is connected between at least one interface unit and at least one accelerator, used to transmit accelerator instructions to the corresponding accelerator or to transmit accelerator instruction responses to the corresponding interface register. The first bus network enables a flexible number of accelerators to be bus-connected to a flexible number of main processors, thereby adapting to a wider variety of machine learning models and artificial intelligence learning tasks, and improving the flexibility of computing power configuration.

[0049] Specifically, the interface unit may also include an interface decoder 134. A main processor may correspond to one or more interface units, and an interface unit may include one or more interface registers. Through the bus front end and bus back end of the first bus network, each accelerator may correspond to at least one specific interface register, or each accelerator may correspond to various interface registers.

[0050] In other examples, the first bus network includes at least one bus front end and at least one bus back end, with the at least one bus front end corresponding to at least one interface register and the at least one bus back end corresponding to at least one accelerator. The first bus network is used to transmit accelerator instructions to the corresponding accelerator or to transmit responses to accelerator instructions to the corresponding interface register via routing between the at least one bus front end and the at least one bus back end.

[0051] Figures 3A-3E for Figure 2 Schematic block diagrams of various variations of the processing system in the embodiments.

[0052] exist Figure 3A In the example, at least one main processor 311 includes main processor 1, main processor 2, and main processor 3. At least one interface unit 312 includes interface unit 1, interface unit 2, and interface unit 3. At least one accelerator 316 includes accelerator 1, accelerator 2, accelerator 3, and accelerator 4.

[0053] At least one bus front end 313 includes the respective bus front end 313 shown, and at least one bus back end 315 includes the respective bus back end 315 shown. In this example, a switch 314 is provided between the respective bus front ends 313 and the respective bus back ends 315. The switch may be a so-called crossbar. The switch 314 is used to route accelerator commands, or responses to accelerator commands, between at least one bus front end and at least one bus back end.

[0054] Specifically, each accelerator can correspond to a specific interface register via a switch in the first bus network.

[0055] exist Figure 3B In the example, at least one main processor includes a first main processor 321 and a second main processor 3210, the first main processor 321 including main processor 1 and main processor 2, and the second main processor 3210 including main processor 3 and processor 4. At least one interface unit includes a first interface unit 322 and a second interface unit 3220, the first interface unit 322 including interface unit 1 and interface unit 2, and the second interface unit 3220 including interface unit 3 and interface unit 4. At least one accelerator includes a first accelerator 326 and a second accelerator 3260, the first accelerator 326 including accelerator 1, accelerator 2, and accelerator 3, and the second accelerator 3260 including accelerator 4.

[0056] In a first bus network including a switch, at least one bus front end 323 includes the illustrated respective bus front ends 323, and at least one bus back end 325 includes the illustrated respective bus back ends 325. In this example, a switch 324 is disposed between the respective bus front ends 323 and the respective bus back ends 325. The switch may be a so-called crossbar. The switch 324 is used to route accelerator commands, or responses to accelerator commands, between at least one bus front end and at least one bus back end. Furthermore, a second bus network 3240 excluding the switch is connected between the second interface unit and the second accelerator.

[0057] Specifically, each accelerator can correspond to at least one specific interface register via the second bus network. In this example, interface unit 3 and interface unit 4 correspond to accelerator 4.

[0058] Without loss of generality, the first bus network includes a first sub-network and a second sub-network; at least one main processor includes a first main processor and a second main processor; at least one interface unit includes a first interface unit corresponding to the first processor and a second interface unit corresponding to the second main processor; the interface register of the first interface unit is connected to at least one accelerator through the first sub-network; and the interface register of the second interface unit is connected to at least one accelerator through the second sub-network. The first bus network includes a switch, while the second bus network does not, further improving the architectural flexibility and making it more suitable for complex and varied AI models and tasks.

[0059] exist Figure 3CIn the example, at least one main processor includes a first main processor 331 and a second main processor 3310, the first main processor 331 including main processor 1 and main processor 2, and the second main processor 3310 including main processor 3. At least one interface unit includes a first interface unit 332 and a second interface unit 3320, the first interface unit 332 including interface unit 1 and interface unit 2, and the second interface unit 3320 including interface unit 3. At least one accelerator includes a first accelerator 336 and a second accelerator 3360, the first accelerator 336 including accelerator 1, accelerator 2, and accelerator 3, and the second accelerator 3360 including accelerator 4 and accelerator 5.

[0060] In a first bus network including a switch, at least one bus front end 333 includes the illustrated respective bus front ends 333, and at least one bus back end 335 includes the illustrated respective bus back ends 335. In this example, a switch 334 is disposed between the respective bus front ends 333 and the respective bus back ends 335. The switch may be a so-called crossbar. The switch 334 is used to route accelerator commands, or responses to accelerator commands, between at least one bus front end and at least one bus back end. Furthermore, a second bus network 3340 excluding the switch is connected between the second interface unit and the second accelerator.

[0061] Specifically, multiple accelerators can correspond to a single interface register via the second bus network. In this example, interface unit 3 corresponds to accelerator 4 and accelerator 5.

[0062] exist Figure 3D In the example, at least one main processor includes a first main processor 341 and a second main processor 3410, where the first main processor 341 includes main processor 1 and main processor 2, and the second main processor 3410 includes main processor 3 and processor 4. At least one interface unit includes a first interface unit 342 and a second interface unit 3420, where the first interface unit 342 includes interface unit 1 and interface unit 2, and the second interface unit 3420 includes interface unit 3 and interface unit 4. At least one accelerator includes a first accelerator 346 and a second accelerator 3460, where the first accelerator 346 includes accelerator 1 and accelerator 2, and the second accelerator 3460 includes accelerator 3.

[0063] Furthermore, a first bus network 344, excluding the switch, is connected between the first interface unit and the first accelerator. Specifically, when the first bus network does not include the switch, a particular interface unit corresponds to a particular accelerator. That is, in this example, interface unit 1 corresponds to accelerator 1, and interface unit 2 corresponds to accelerator 2.

[0064] The second bus network 3440, excluding the switch, is connected between the second interface unit and the second accelerator.

[0065] exist Figure 3E In the example, at least one main processor includes a first main processor 351 and a second main processor 3510, the first main processor 351 including main processor 1 and main processor 2, and the second main processor 3510 including main processor 3. At least one interface unit includes a first interface unit 352 and a second interface unit 3520, the first interface unit 352 including interface unit 1 and interface unit 2, and the second interface unit 3520 including interface unit 3. At least one accelerator includes a first accelerator 356 and a second accelerator 3560, the first accelerator 356 including accelerator 1 and accelerator 2, and the second accelerator 3560 including accelerator 3 and accelerator 4.

[0066] Furthermore, a first bus network 354, excluding the switch, is connected between the first interface unit and the first accelerator. A second bus network 3540, excluding the switch, is connected between the second interface unit and the second accelerator.

[0067] Without loss of generality, the first bus network includes a first subnetwork and a second subnetwork, and at least one accelerator includes a first accelerator and a second accelerator. An interface register is connected to the first accelerator via the first subnetwork, and the interface register is connected to the second accelerator via the second subnetwork. Different accelerators can be connected in parallel to the same main processor via the first and second subnetworks, or different main processors can be connected to accelerators via the first and second subnetworks, further improving the architectural flexibility and making it more suitable for complex and varied AI models and tasks.

[0068] In other examples, at least one main processor includes multiple main processors, at least one accelerator includes multiple accelerators, and a first bus network includes multiple sub-networks. The multiple main processors, multiple accelerators, and multiple sub-networks are respectively corresponded, with each main processor connected to its corresponding accelerator via a corresponding sub-network. Multiple sub-networks further enhance the flexibility of the architecture, making it more suitable for complex and varied AI models and tasks.

[0069] exist Figure 2 Examples and Figures 3A-3EIn the various specific examples, the main processor communicates with the corresponding accelerators through its interface units. There is a specific correspondence between the main processor and the interface units, which differs from the correspondence between the interface units and the accelerators. Specifically, the correspondence between interface units and accelerators is one-to-one when the first bus network does not include switches. When the first bus network includes switches, the correspondence can be many-to-many, meaning each interface unit may transmit data to any accelerator for processing. In the case of the second bus, the correspondence between interface units and accelerators can be one-to-many or many-to-one.

[0070] Furthermore, when the first bus network does not include a switch and the correspondence between interface units and accelerators is many-to-one, the queue management of the interface units can adopt... Figure 1 The queue management method allows multiple interface units to allocate their respective data to the same accelerator for processing.

[0071] In the case where the first bus network includes a switch, the switch can merge and manage multiple queues of each interface unit. That is, the output queues and input queues of each interface unit are managed in a unified manner. When retrieving data to be processed from the output queue of an interface unit, the data to be processed is allocated based on the current load of each accelerator (i.e., the length of the data queue waiting for an accelerator to process). For example, the data to be processed of an interface unit is allocated to the accelerator with the lowest current load for processing.

[0072] Furthermore, in one example, the switch can also map the response to an accelerator instruction to the grounding unit that generated the accelerator instruction, and the interface unit returns the response to the corresponding main processor for further instruction processing. In another example, the main processor that generates the accelerator instruction is a scheduling processor, and the accelerator instruction carries the identifier of the executing processor, rather than the identifier of the scheduling processor that generated the accelerator instruction. The switch can map the response to the accelerator instruction to the interface unit of the executing processor, and the interface unit returns the response to the corresponding executing processor for further instruction processing.

[0073] Figure 4 This is a schematic block diagram of a processing system according to another embodiment of the present invention. The processing system of this embodiment includes a first main processor 410, a first interface unit 420, a second main processor 430, a second interface unit 440, and a third bus network 450.

[0074] The first main processor 410 is used to output the first processor instructions.

[0075] The first interface unit 420 is configured correspondingly to the first main processor. The first interface unit 420 includes an interface front-end 421 and an interface register 422. The interface front-end 421 is connected to the first main processor and is used to receive instructions from the first processor, generate transmission instructions corresponding to the processor instructions, and send the transmission instructions. The interface register 422 is used to receive the sent transmission instructions and forward the transmission instructions or receive responses to the transmission instructions.

[0076] The second main processor 430 is used at least to process the instructions of the second processor and obtain the processing results of the instructions of the second processor.

[0077] The second interface unit 440 is configured correspondingly to the second main processor. The second interface unit 440 includes an interface register 441 and an interface front-end 442. The interface register 441 is used to receive transmitted transmission commands and forward transmission commands or receive responses to transmission commands. The interface front-end 442 is connected to the second main processor and is used to receive transmission commands, generate second processor instructions corresponding to the transmission commands, or generate responses to transmission commands based on the processing results of the second processor instructions.

[0078] The third bus network 450 is connected between the first main processor and the second main processor, and is used to transmit transmission instructions from the first main processor to the second main processor, or to transmit the response to the transmission instructions from the second main processor to the first main processor.

[0079] In the processing system according to the embodiments of this application, a first interface unit is configured corresponding to a first main processor, and a second interface unit is configured corresponding to a second main processor. Instruction conversion between the first and second main processors is achieved through the interface front-end. The interface register is used to forward the converted instructions, which facilitates instruction queue management. Thus, second processor instructions suitable for processing by the second main processor are obtained through the interface unit. Furthermore, a third bus network is connected between the first and second interface units, used to transmit second processor instructions to the second main processor or to transmit responses to second processor instructions to the corresponding interface register. The third bus network enables a flexible number of main processors to be connected via a bus, making it suitable for a wider variety of machine learning models and artificial intelligence learning tasks, and improving the flexibility of computing power configuration.

[0080] Specifically, the interface units of different main processors are connected through a third bus network. The processor instructions of a specific main processor can be converted into a general processor format (i.e., transmission instructions) and then routed through the third bus network. The interface units of other main processors convert the transmission instructions into their own processor instructions and forward them to their main processors for processing.

[0081] Furthermore, heterogeneous computing between each main processor and accelerator (e.g., transfers based on a first bus network or a second bus network) can be referenced. Figure 2 as well as Figures 3A-3E Various embodiments thereof.

[0082] Figures 5A-5D for Figure 4 Schematic block diagrams of various variations of the processing system in the embodiments.

[0083] exist Figure 5A In the example, the first main processor 511 includes main processor 1, and the first interface unit 512 includes interface unit 1. The second main processor 513 includes main processor 2, main processor 3, and main processor 4, and the second interface unit 514 includes interface unit 2, interface unit 3, and interface unit 4. The third bus network also includes multiple sub-networks 515, respectively connected between the first main processor 511 and the second main processor 513. Furthermore, Figure 5A The processing system also includes a fourth bus network 5150 and at least one accelerator 516 (accelerator 1).

[0084] Specifically, different main processors can assist other main processors in executing instructions, and accelerators can assist any main processor connected to them in executing instructions.

[0085] exist Figure 5B In the example, the first main processor 521 includes main processor 1, and the first interface unit 522 includes interface unit 1. The second main processor 523 includes main processor 2, and the second interface unit 524 includes interface unit 2. A third bus network 525 is connected between interface unit 1 and interface unit 2. In addition, a fourth bus network includes multiple sub-networks 5250, which are respectively disposed between interface unit 1 and each accelerator 526.

[0086] Without loss of generality, the first main processor is also used to output second processor instructions. The interface front end of the first interface unit is also used to receive second processor instructions, generate second accelerator instructions corresponding to the second processor instructions, and send the second accelerator instructions. The interface register of the first interface unit is also used to receive the sent second accelerator instructions and forward the second accelerator instructions or receive responses to the second accelerator instructions. The processing system further includes: at least one accelerator for processing the forwarded second accelerator instructions and obtaining responses to the second accelerator instructions; and a first bus network connected between the first interface unit and at least one accelerator for transmitting second accelerator instructions to the corresponding accelerator or transmitting responses to the second accelerator instructions to the interface register of the first interface unit. The first bus network enables a flexible number of main processors and accelerators to be connected via a bus, thus adapting to a wider variety of machine learning models and artificial intelligence learning tasks, and improving the flexibility of computing power configuration.

[0087] exist Figure 5C In the example, the first main processor 531 includes main processor 1, and the first interface unit 532 includes interface unit 1. The second main processor 533 includes main processor 2, main processor 3, and main processor 4, and the second interface unit 534 includes interface unit 2, interface unit 3, and interface unit 4. The third bus network also includes multiple sub-networks 535, respectively connected between the first main processor 531 and the second main processor 533. Furthermore, Figure 5A The processing system also includes a fourth bus network 5350 and at least one accelerator 536 (accelerator 1). Each sub-network 5350 is connected between accelerator 2 and interface unit 2, accelerator 3 and interface unit 3, and accelerator 4 and interface unit 4, respectively.

[0088] Without loss of generality, the second main processor is also used to output third processor instructions. The interface front end of the second interface unit is also used to receive third processor instructions, generate third accelerator instructions corresponding to the third processor instructions, and send the third accelerator instructions. The interface register of the second interface unit is also used to receive the sent third accelerator instructions and forward the third accelerator instructions or receive responses to the third accelerator instructions. Furthermore, the processing system may also include: at least one accelerator for processing the forwarded third accelerator instructions and obtaining responses to the third accelerator instructions; and a fourth bus network connected between the second interface unit and at least one accelerator for transmitting third accelerator instructions to the corresponding accelerator or transmitting responses to the third accelerator instructions to the interface register of the second interface unit. The fourth bus network enables a flexible number of main processors and accelerators to be connected via bus, thus adapting to a wider variety of machine learning models and artificial intelligence learning tasks, and improving the flexibility of computing power configuration.

[0089] Specifically, the interface unit converts instructions into corresponding commands based on the bus network to which it is connected. For example, when connected to an accelerator, the interface unit performs the conversion between the main processor's processor instructions and the accelerator instructions; when connected to other main processors, the interface unit performs the conversion between the main processor's processor instructions and those of other main processors.

[0090] exist Figure 5D In the example, the first main processor 541 includes a scheduling processor 1, and the first interface unit 542 includes an interface unit 1. The second main processor 543 includes execution processors 2, 3, and 4, and the second interface unit 544 includes interface units 2, 3, and 4. The third bus network also includes multiple sub-networks 545, respectively connected between the scheduling processor 541 and the execution processor 543. Furthermore, Figure 5A The processing system also includes a fourth bus network 5450 and at least one accelerator 546 (accelerator 1). Each sub-network 5450 is connected between accelerator 2 and interface unit 2, accelerator 3 and interface unit 3, and accelerator 4 and interface unit 4, respectively.

[0091] Furthermore, when the first main processor is a scheduling processor and the second main processor is an execution processor, the scheduling processor reads instructions from main memory, converts the read instructions into execution instructions for the execution processor to process, and the accelerator executes the received instructions from the execution main processor or the scheduling processor.

[0092] Figure 6 This is a flowchart illustrating the steps of an instruction transmission method according to another embodiment of the present invention. The instruction transmission method of this embodiment is applied to an interface unit, which can be any of the interface units described in the above embodiments. The instruction transmission method includes:

[0093] S610: Receives processor instructions from the main processor connected to the interface unit.

[0094] S620: Generates accelerator instructions corresponding to processor instructions.

[0095] S630: Determine the target accelerator corresponding to the accelerator command from among multiple accelerators connected to the first bus network.

[0096] S640: Sends accelerator commands to the target accelerator via the first bus network.

[0097] In the processing system according to the embodiments of this application, the first bus network realizes a flexible correspondence between the interface unit of the main processor and each accelerator, improves the flexibility of instruction scheduling, and thus realizes the acceleration efficiency of the accelerator.

[0098] In another example, determining the target accelerator corresponding to an accelerator instruction from multiple accelerators connected to a first bus network includes: determining the number of instructions to be processed for each of the multiple accelerators connected to the first bus network; and determining the target accelerator among the multiple accelerators such that the number of instructions to be processed for the target accelerator is less than the number of instructions to be processed for the other accelerators. In this example, the number of instructions to be processed for the target accelerator is less than the number of instructions to be processed for the other accelerators, thereby further improving the overall acceleration efficiency of the accelerator.

[0099] In another example, sending accelerator commands to the target accelerator via a first bus network includes: sending the accelerator commands and the target accelerator's identifier to a switch on the first bus network via the bus front-end corresponding to the interface unit; the identifier instructing the switch to locate the target accelerator; and forwarding the accelerator commands to the target accelerator. In this example, a more flexible command forwarding is achieved through a switch in the first bus network.

[0100] In another example, the instruction transmission method further includes receiving a processing response of the accelerator instruction returned by the target accelerator via a first bus network. In this example, the first bus network sends the accelerator instruction and receives the processing response of the accelerator instruction, which improves the instruction transmission efficiency and saves transmission costs.

[0101] Reference Figure 7 The diagram shows a schematic of an electronic device according to another embodiment of the present invention. The specific embodiments of the present invention do not limit the specific implementation of the electronic device.

[0102] like Figure 7 As shown, the electronic device may include: a processor 702, a communications interface 704, a memory 706, and a fifth bus network 708. The fifth bus network 708 may be an internal bus network, and the fifth bus network may be the same as or different from the first bus network.

[0103] The processing system 702, communication interface 704, and memory 706 communicate with each other via a fifth bus network 708. The communication interface 704 is used to communicate with other electronic devices or servers.

[0104] The processing system 702 is used to execute program 710, specifically the relevant steps in the above method embodiments.

[0105] Specifically, program 710 may include program code that includes computer operation instructions.

[0106] The processing system 702 may be a processor CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0107] Memory 706 is used to store program 710. Memory 706 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0108] Specifically, program 710 can be used to cause the order receiving unit of processing system 702 to execute... Figure 6 The method of transmitting instructions.

[0109] Furthermore, the specific implementation of each step in the program 710 stored in the memory can be found in the corresponding descriptions of the steps and units in the above method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0110] Furthermore, embodiments of the present invention also provide a computer storage medium having a computer program stored thereon, which is executed by the processing system according to the various embodiments described above.

[0111] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0112] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA)). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., Random Access Memory (RAM), Read-Only Memory (ROM), Flash Memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0113] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0114] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A processing system, comprising: At least one main processor for outputting processor instructions, the at least one main processor including a first main processor and a second main processor, the first main processor being a scheduling processor and the second main processor being an execution processor; At least one interface unit is configured corresponding to the at least one main processor, the interface unit comprising: The interface front end is connected to the main processor and is used to receive processor instructions, generate processor instructions, and send accelerator instructions; An interface register is used to receive transmitted accelerator commands and forward the accelerator commands or receive responses to the accelerator commands. The interface register includes an input queue and an output queue. At least one accelerator is used to process forwarded accelerator instructions and obtain a response to the accelerator instructions; A first bus network includes a switch, at least one bus front end, and at least one bus back end. The at least one bus back end is configured to correspond to at least one accelerator and is used to transmit the accelerator instructions to the corresponding accelerator. The at least one bus front end is configured to correspond to at least one interface register and is used to transmit the response of the accelerator instructions to the corresponding interface register. The switch is located between the at least one bus front end and the at least one bus back end. The main processor that generates the accelerator instructions is a scheduling processor. The accelerator instructions carry the identifier of the execution processor. The switch maps the response of the accelerator instructions to the interface unit of the execution processor, and the interface unit returns the response to the corresponding execution processor for further instruction processing. The switch also retrieves data to be processed from the output queue of an interface unit and distributes the data to be processed based on the current load of each accelerator, wherein the length of the data queue waiting to be processed by an accelerator indicates the current load of that accelerator.

2. The processing system according to claim 1, wherein, The first bus network includes a first sub-network and a second sub-network, and the at least one interface unit includes a first interface unit corresponding to the first main processor and a second interface unit corresponding to the second main processor; The interface register of the first interface unit is connected to the at least one accelerator through the first sub-network, and the interface register of the second interface unit is connected to the at least one accelerator through the second sub-network.

3. The processing system according to claim 1, wherein, The at least one main processor includes multiple main processors, the at least one accelerator includes multiple accelerators, and the first bus network includes multiple sub-networks. The plurality of main processors, the plurality of accelerators, and the plurality of sub-networks are respectively corresponding, and the main processors among the plurality of main processors are connected to the corresponding accelerators through the corresponding sub-networks.

4. A processing system, comprising: The first main processor is used to output first processor instructions, and the first main processor is a scheduling processor; A first interface unit is configured corresponding to the first main processor, and the first interface unit includes: The interface front end is connected to the scheduling processor and is used to receive the first processor instruction, generate the transmission instruction corresponding to the processor instruction, and send the transmission instruction. An interface register is used to receive transmitted transmission instructions and forward the transmission instructions or receive responses to the transmission instructions. The interface register includes an input queue and an output queue. The second main processor is used at least to process the instructions of the second processor and obtain the processing results of the instructions of the second processor. The second main processor is the execution processor. A second interface unit is configured corresponding to the second main processor, and the second interface unit includes: An interface register is used to receive transmitted transmission instructions and forward the transmission instructions or receive responses to the transmission instructions. The interface register includes an input queue and an output queue. The interface front end is connected to the execution processor and is used to receive the transmission instruction, generate a second processor instruction corresponding to the transmission instruction, or generate a response to the transmission instruction based on the processing result of the second processor instruction; A third bus network is connected between the first main processor and the second main processor for transmitting the transmission command from the first main processor to the second main processor, or transmitting the response to the transmission command from the second main processor to the first main processor. At least one accelerator is used to process the forwarded accelerator commands and obtain a response to the accelerator commands. A first bus network includes a switch, at least one bus front end, and at least one bus back end. The at least one bus back end is configured to correspond to at least one accelerator and is used to transmit accelerator instructions generated by a scheduling processor and carrying the identifier of the execution processor to the corresponding accelerator. The at least one bus front end is configured to correspond to at least one interface register and is used to transmit the response of the accelerator instruction to the corresponding interface register. The switch is configured between the at least one bus front end and the at least one bus back end and is used to map the response of the accelerator instruction to a second interface unit of the execution processor, from which the second interface unit returns to the corresponding execution processor for further instruction processing. The switch also retrieves data to be processed from the output queue of an interface unit and distributes the data to be processed based on the current load of each accelerator, wherein the length of the data queue waiting to be processed by an accelerator indicates the current load of that accelerator.

5. The processing system according to claim 4, wherein, The first main processor is also used to output instructions to the second processor; The interface front end of the first interface unit is also used to receive the second processor instruction, generate the second accelerator instruction corresponding to the second processor instruction, and send the second accelerator instruction; The interface register of the first interface unit is also used to receive the sent second accelerator instruction, and to forward the second accelerator instruction or receive the response of the second accelerator instruction.

6. The processing system according to claim 4, wherein, The second main processor is also used to output instructions to the third processor; The interface front end of the second interface unit is also used to receive the third processor instruction, generate the third accelerator instruction corresponding to the third processor instruction, and send the third accelerator instruction. The interface register of the second interface unit is also used to receive the sent third accelerator command, and to forward the third accelerator command or receive the response of the third accelerator command; The processing system also includes: At least one accelerator is used to process the forwarded third accelerator command and obtain a response to the third accelerator command; A fourth bus network, connected between the second interface unit and the at least one accelerator, is used to transmit the third accelerator instruction to the corresponding accelerator or to transmit the response of the third accelerator instruction to the interface register of the second interface unit.

7. A method for transmitting instructions, comprising: The interface unit receives processor instructions from a scheduling processor, which is a first main processor connected to the interface unit. The interface unit includes an interface front-end and an interface register, and the interface register includes an input queue and an output queue. The processor instruction is generated to correspond to an accelerator instruction that carries the identifier of the execution processor, which serves as the second main processor. From a plurality of accelerators connected to a first bus network, determine the target accelerator corresponding to the accelerator instruction; Sending accelerator instructions to the target accelerator via the first bus network includes: determining the number of instructions to be processed for each of the multiple accelerators connected to the first bus network. Determine a target accelerator among the plurality of accelerators, such that the number of pending instructions for the target accelerator is less than the number of pending instructions for the other accelerators; The response of the accelerator instruction obtained by the target accelerator processing the accelerator instruction is mapped to the interface unit of the execution processor through the switch in the first bus network, so that the interface unit can return to the corresponding execution processor to perform further instruction processing.

8. An electronic device, comprising: According to any one of claims 1-6, the processing system, the memory, the communication interface, and the fifth bus network, the processing system, the memory, and the communication interface communicate with each other through the fifth bus network; The memory is used to store at least one executable instruction of a computer program, and the main processor of the processing system generates processor instructions based on the at least one executable instruction.

9. A computer storage medium having a computer program stored thereon, the program being executed by a processing system according to any one of claims 1-6.

Images