Instruction processing method, scheduler, processor, chip and electronic equipment

Abstract

The invention discloses an instruction processing method, a scheduler, a processor, a chip and electronic equipment, and relates to the field of computers. Executing the first scoreboard checking instruction, checking whether enough idle resources in the scoreboard support the configuration of the to-be-processed sub-graph, if yes, executing the sub-graph configuration instruction to configure the to-be-processed sub-graph in the scoreboard, executing the second scoreboard checking instruction to check whether the to-be-processed sub-graph completes the configuration in the scoreboard, and if yes, executing the sub-graph configuration instruction to configure the to-be-processed sub-graph in the scoreboard; and if so, scheduling and controlling whether to release the plurality of to-be-processed operators according to the configuration information of the to-be-processed sub-graphs recorded in the scoreboard. According to the method, the sub-graph is configured in the scoreboard, and the input cache information and the output cache information corresponding to each operator and various counters related to the output cache are recorded in the scoreboard, so that related data storage information can be represented concisely and clearly to serve as a reference basis for whether the operator in the sub-graph is released or not; and the data dependence among the operators can be well solved.

Description

Technical Field

[0001] This application relates to the field of computers, and more specifically, to an instruction processing method, a scheduler, a processor, a chip, and an electronic device. Background Technology

[0002] In current microprocessor architectures, concurrent and out-of-order instruction execution are common practices to improve overall performance. One problem arising from concurrent and out-of-order instruction execution is the need to resolve data dependencies between instructions. However, current methods for resolving these data dependencies are not very effective. Summary of the Invention

[0003] In view of the above problems, this application proposes an instruction processing method, scheduler, processor, chip, and electronic device to solve the above problems.

[0004] In a first aspect, embodiments of this application provide an instruction processing method applied to a scheduler. The method includes: executing a first scoreboard check instruction to check whether there are sufficient idle resources in the scoreboard to support the configuration of a subgraph to be processed; if it is found that there are sufficient idle resources in the scoreboard to support the configuration of the subgraph to be processed, then executing a subgraph configuration instruction to configure the subgraph to be processed in the scoreboard, wherein the subgraph configuration instruction is determined based on the data dependency relationship between multiple operators to be processed included in the subgraph to be processed; executing a second scoreboard check instruction to check whether the subgraph to be processed has been configured in the scoreboard; if it is found that the subgraph to be processed has been configured in the scoreboard, then scheduling control is used to determine whether to allow the multiple operators to be processed based on the configuration information of the subgraph to be processed recorded in the scoreboard, wherein the configuration information of the subgraph to be processed includes input buffer information, output buffer information, and various counters related to the output buffer for each of the multiple operators to be processed.

[0005] Secondly, embodiments of this application provide a scheduler for executing the above-described method.

[0006] Thirdly, embodiments of this application provide a processor that includes the scheduler described above.

[0007] Fourthly, embodiments of this application provide an electronic device that includes the processor described above.

[0008] Fifthly, embodiments of this application provide a chip including the aforementioned processor.

[0009] The instruction processing method, scheduler, processor, chip, and electronic device provided in this application first execute a first scoreboard check instruction to check whether there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed. If there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed, a subgraph configuration instruction is executed to configure the subgraph to be processed in the scoreboard. The subgraph configuration instruction is determined based on the data dependencies between the multiple operators to be processed included in the subgraph to be processed. Then, a second scoreboard check instruction is executed to check whether the subgraph to be processed has been configured in the scoreboard. If the subgraph to be processed has been configured in the scoreboard, the configuration is performed according to the records in the scoreboard. The configuration information of the subgraph to be processed is used to schedule and control whether to allow multiple operators to be processed. The configuration information of the subgraph to be processed includes the input buffer information, output buffer information, and various counters related to the output buffer for each operator. Thus, the subgraph is configured in the scoreboard according to the data dependencies between the operators in the subgraph, and the input buffer information, output buffer information, and various counters related to the output buffer for each operator are recorded in the scoreboard. This can concisely and clearly represent the relevant data storage information, which can serve as a reference for whether to allow the operators in the subgraph to be processed. This can effectively solve the data dependencies between operators and improve processing efficiency. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A general architecture diagram of the instruction processing method provided in an embodiment of this application is shown;

[0012] Figure 2 A flowchart illustrating an embodiment of the instruction processing method provided in this application is shown.

[0013] Figure 3 This illustration shows a schematic diagram of the branch data cache provided in an embodiment of this application;

[0014] Figure 4 A flowchart illustrating an embodiment of the instruction processing method provided in this application is shown.

[0015] Figure 5 A flowchart illustrating an embodiment of the instruction processing method provided in this application is shown.

[0016] Figure 6 This illustration shows a schematic diagram of the scoreboard configuration process provided in an embodiment of this application;

[0017] Figure 7 A flowchart illustrating an embodiment of the instruction processing method provided in this application is shown.

[0018] Figure 8 This application provides configuration-execution diagrams of different sub-graphs in its embodiments.

[0019] Figure 9 This illustration shows a comparison of the hidden and unhidden subgraph configuration instructions provided in an embodiment of this application;

[0020] Figure 10 A flowchart illustrating an embodiment of the instruction processing method provided in this application is shown.

[0021] Figure 11 This application provides a consumer-producer schematic diagram based on an embodiment of the present application.

[0022] Figure 12 This application illustrates a scheme for calculating data volume using a counter in a scoreboard, as provided by an embodiment of the present application.

[0023] Figure 13 This application shows a topological diagram of the subgraph provided in an embodiment.

[0024] Figure 14 A schematic diagram of the subgraph execution flow provided in an embodiment of this application is shown. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0026] Out-of-order processor architectures improve instruction execution efficiency and performance by allowing independent instructions to be executed out of order. However, there are often data dependencies between instructions. Therefore, a key issue that out-of-order processor architectures need to address is instruction scheduling and distribution, requiring that the execution order of instructions strictly adhere to their corresponding data dependencies.

[0027] The scoreboard technology was first proposed and implemented in the CDC 6600 and is now widely used in current architectures such as RISC-V. Previously, scoreboard technology was often used to implement hardware dynamic instruction scheduling. Scoreboard technology includes a series of registers and hardware circuitry that controls instruction issuance and scheduling. It can be understood that the scoreboard is a representation of information stored in memory. By recording the data loading status of registers on the scoreboard, it controls instruction access to registers. For example, when an instruction needs to read or write to a register, it can check the data loading status of that register recorded on the scoreboard to ensure that instruction execution does not cause data conflicts. The scoreboard first checks for structural hazards and data hazards for each instruction set during the decoding stage and generates relevant records, which are stored in registers. Then, based on these records, it determines whether the instruction can be issued for execution or needs to be paused. Simultaneously, the scoreboard monitors the availability of operands for relevant instructions to control when relevant instructions can read registers. Finally, the scoreboard controls the operation of writing output data to registers. The introduction of scoreboard technology enables out-of-order and concurrent execution of instructions, greatly improving the execution efficiency of related programs.

[0028] Currently, most Artificial Intelligence (AI) processors often employ homogeneous or heterogeneous approaches to decompose computations in order to achieve parallelization. Homogeneous processing mainly involves multiple engines / sub-engines with the same functional architecture, sharing the overall computational task (e.g., different regions / patches of an image). Heterogeneous processing uses multiple computational engines with different functionalities to implement different computational processing functions, such as DMA engines, scalar computation engines, vector computation engines, matrix computation engines, etc. This leads to the question of how to schedule these multiple engines. In traditional models, scheduling can be categorized into hardware scheduling (dedicated schedulers, serving only specific algorithms / scenes) and software scheduling (flexible). Both scheduling models need to address the issues of data dependencies and data hazards.

[0029] Furthermore, AI processors are often based on vector processor architectures, or even specific tensor processor architectures such as systolic arrays. The amount of data required for a single instruction in an AI processor is significantly higher than that of traditional processors, often relying on vector registers (e.g., in the RV64V, a single vector register is 2048 bits wide, and there are 32 vector registers) or even memory (e.g., most systolic array input data needs to be read directly from SRAM). To resolve these data hazards, a mechanism is required to record and maintain data in input and output caches at a larger granularity and on a wider scale (not limited to registers, but extending to other memories). Simultaneously, considering hardware area, power consumption, and complexity, the corresponding solution needs to concisely and clearly represent the relevant data storage information. Therefore, the traditional approach of allocating a separate bit for each memory location, as in traditional scoreboards, is impractical.

[0030] Meanwhile, in AI neural networks, the introduction of residual topology means that for a fixed data cache, multiple operators may read data, defined as multiple consumers. Conversely, an operator may have multiple output caches, defined as multiple producers. Therefore, the scoreboard mechanism faces new requirements: data hazard detection needs to support one-to-many, many-to-one, or many-to-many producer-consumer relationships.

[0031] To address the aforementioned problems, the inventors, through long-term research, discovered and proposed the instruction processing method, scheduler, processor, chip, and electronic device provided in the embodiments of this application. Based on the data dependencies between operators in the subgraph, the subgraph is configured in a scoreboard, and the input buffer information, output buffer information, and various counters related to the output buffer are recorded in the scoreboard. This provides a concise and clear representation of relevant data storage information, serving as a reference for whether to allow or disallow operators in the subgraph. This effectively resolves the data dependencies between operators and improves processing efficiency. The specific instruction processing method will be described in detail in subsequent embodiments.

[0032] To better understand the solutions of the embodiments of this application, the technical terms used in the embodiments of this application will be explained below.

[0033] SOC (System on Chip): System on a chip

[0034] OP (Operator): Operator

[0035] psv (preserve): Reserved

[0036] wcntr(Write Counter): Write counter

[0037] rcntr (Read Counter): Read counter

[0038] SCB (Scoreboard): Scoreboard

[0039] AI (Artificial Intelligence): Artificial Intelligence

[0040] WAW (Write After Write): Write after writing

[0041] WAR (Write After Read): read after writing

[0042] RAW (Read After Write): Read after writing

[0043] It should be noted that the chip processes tasks using task-matching algorithms (e.g., image processing algorithms). An algorithm can include multiple operators (e.g., the "1+1" operator), and one or more operators form a subgraph. Any operation performed on any function can be considered an operator. An operator corresponds to one or more instructions (e.g., a shift instruction (shift 1), a quantization instruction, a calculation instruction (+)).

[0044] See Figure 1 , Figure 1 The overall architecture diagram of the instruction processing method provided in the embodiments of this application is shown. Figure 1 The processor shown can be a Neural Processing Unit (NPU), a Digital Signal Processor (DSP), a Central Processing Unit (CPU), etc., and is not limited thereto. This processor may include hardware units such as an Instruction Dispatcher, a Scheduler, and a sub-engine. In this embodiment, the instruction processing method can be executed by the scheduler, which may include a scoreboard configuration engine. This scoreboard configuration engine can be used to synchronize the configuration and execution phase instructions on the scoreboard for each subgraph, such as the first scoreboard check instruction, the second scoreboard check instruction, the first scoreboard update instruction, and the second scoreboard update instruction in this embodiment. Of course, the scheduler may also include a callback module, which will not be elaborated further here. Subsequent embodiments will be combined with... Figure 1 The above-mentioned hardware units provide a detailed explanation of the instruction processing methods.

[0045] In this embodiment, the processor may include one or more processing units or sub-computation engines, denoted as sub-engine1, sub-engine2, etc. A sub-engine is defined as a processing unit capable of architecture-level parallelism. For example, while sub-engine1 is processing a block of data, sub-engine2 can synchronously prefetch the next block of data. In AI processors, sub-engines often correspond to parallelizable sub-computation engines, such as DMA sub-computation engines, Vector sub-computation engines, Matrix sub-computation engines, etc., and are not limited here.

[0046] It can be understood that the smallest computing node in each sub-computing engine is an atomic computing node. Each atomic computing node corresponds to its own dedicated input and output data cache space. A single atomic computing node may contain one or more instructions, but the data space involved in these instructions is confined to the data cache space of this atomic computing node. In AI computing, an atomic computing node is often defined as an operator. Due to the limitations of on-chip memory, in AI processors, a set of a series of atomic computing nodes is often defined as a subgraph. Data processing of a subgraph can often be completed in on-chip memory.

[0047] in addition, Figure 1 A scoreboard is used to configure, record, and update the configuration information of subgraphs (which include multiple operators). This includes information such as consumer information, producer information, total data space information, and remaining free space information for the corresponding data, which are not limited here.

[0048] The following will describe the relevant instructions involved in the embodiments of this application. In the embodiments of this application, the relevant instructions are divided into four main categories:

[0049] The first category consists of configuration and execution commands required by each sub-engine during execution, denoted as configcommand and execution command, or simply CFG and EXEC commands. CFG and EXEC commands are defined by each sub-engine based on its actual functionality and will not be elaborated upon here.

[0050] The second category consists of Scoreboard configuration directives for each of the multiple operators included in a subgraph, known as SCBOP config directives.

[0051] The third category consists of instructions used to check data dependencies between operators and control the issuance and data retrieval of operator instructions. In this application's embodiments, operator check instructions (operator CHECK instructions), data reservation instructions (operator PRESERVE instructions), and operator update instructions (operator UPDATE instructions) are used to implement these functions. In specific implementations, the relevant instructions are not limited to these three instructions; any instruction that achieves the corresponding function is acceptable.

[0052] The fourth category: Due to the limitations of on-chip memory size, the software layer often defines a set of operator nodes in the AI ​​model as a subgraph. For each subgraph, the instructions that synchronize its configuration phase on the scoreboard with the subgraph execution phase are called the Scoreboard Check instruction (SCB CHECK instruction) and the Scoreboard Update instruction (SCB UPDATE instruction).

[0053] Please see Figure 2 , Figure 2 This diagram illustrates a flowchart of an instruction processing method provided in an embodiment of this application. The method configures the subgraph in a scoreboard based on the data dependencies between operators within the subgraph, and records the input buffer information, output buffer information, and various counters related to the output buffer for each operator in the scoreboard. This provides a concise and clear representation of the relevant data storage information, serving as a reference for whether to allow or disallow operations in the subgraph. This effectively resolves data dependencies between operators and improves processing efficiency. In a specific embodiment, this method can be applied to a scheduler. The following will focus on... Figure 2 The process shown will be described in detail. The instruction processing method may specifically include the following steps:

[0054] Step S110: Execute the first scoreboard check instruction to check whether there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed.

[0055] A compiler is a software tool that converts source code written in high-level programming languages ​​(such as C, C++, Java, etc.) into low-level code (such as machine language or bytecode) that a computer can directly execute. The compiler primarily generates instructions for hardware (such as the scheduler in this embodiment) to execute, and inserts them into corresponding positions to form instruction sequences for subsequent hardware processing. Optionally, in this embodiment, the compiler can be used to generate the second, third, and fourth types of instructions mentioned above. Optionally, the compiler can work offline; that is, it can be independent of the hardware and can compile relevant instructions into instruction sequences in advance on other hardware platforms (such as servers or computers). Subsequently, the scheduler can obtain the instruction sequence for corresponding execution.

[0056] The compiler can segment subgraphs according to the on-chip memory specifications of the overall AI processor, inserting corresponding scoreboard check instructions before each subgraph enters the configuration phase and before each subgraph enters the execution phase. In this embodiment, for the segmented subgraphs to be processed, a first scoreboard check instruction can be inserted before the subgraph enters the configuration phase, and a second scoreboard check instruction can be inserted before the subgraph enters the execution phase.

[0057] The compiler can segment subgraphs according to the on-chip memory specifications of the overall AI processor. For each subgraph, it can generate corresponding subgraph configuration instructions based on the data dependencies between the multiple operators it includes. In this embodiment, for the segmented subgraphs to be processed, the corresponding subgraph configuration instructions are inserted when the subgraph enters the configuration stage. It is understood that during the compilation stage of the subgraph to be processed, the compiler can fully see the data flow graph and data dependency graph corresponding to the multiple operators included in the subgraph. Therefore, for the subgraph to be compiled, the compiler can generate corresponding subgraph configuration instructions based on the data flow graph and data dependency graph.

[0058] Therefore, for the subgraph to be processed, the compiler can generate the corresponding first scoreboard check instruction, second scoreboard check instruction, and subgraph configuration instruction and insert them accordingly to obtain an instruction sequence, so that the subsequent hardware can perform corresponding operations based on the instruction sequence.

[0059] In this embodiment, the scheduler can obtain the instruction sequence compiled by the compiler. Optionally, the compiler can store the instruction sequence after obtaining it. It is understood that after storing the instruction sequence, the hardware used to execute the function instructions (such as the scheduler) can obtain the instruction sequence. As one approach, the instruction dispatch unit can read the instruction sequence stored by the compiler and send it to the scheduler, which can then obtain the instruction sequence.

[0060] In some implementations, when a compiler obtains a sequence of instructions, it can store it in the target address space of the other hardware platform on which it runs. Accordingly, the instruction dispatch unit and scheduler can retrieve the instruction sequence from the target address space. For example, if the compiler runs on a server, it can store the instructions in the server's target address space; if the compiler runs on a computer, it can store the instructions in the computer's target address space. Alternatively, when a compiler obtains a sequence of instructions, it can store it in the target address space of a server connected to the other hardware platform on which it runs. For example, if the compiler runs on a computer, it can store the instructions in the target address space of a server connected to the computer, and so on. This is not a limitation.

[0061] Therefore, in this embodiment, the construction of relevant data dependencies can be generated by the compiler, and the hardware is only responsible for recording and updating, which greatly simplifies the hardware design and can reduce the hardware area and power consumption.

[0062] Optionally, the first scoreboard check instruction (first SCB CHECK instruction) can be used to determine whether there are sufficient free scoreboard resources. In this embodiment, the scheduler can execute the first scoreboard check instruction to check whether there are enough free resources on the scoreboard to support the configuration of the subgraph to be processed. The scheduler may include a scoreboard configuration engine, which can execute the first scoreboard check instruction to check whether there are enough free resources on the scoreboard to support the configuration of the subgraph to be processed.

[0063] In some implementations, the scheduler can determine the resources required by the subgraph to be processed and determine the available resources in the scoreboard. Based on the resources required by the subgraph to be processed and the available resources in the scoreboard, it checks whether there are enough available resources in the scoreboard to support the configuration of the subgraph to be processed. Optionally, if based on the resources required by the subgraph to be processed and the available resources in the scoreboard, it is determined that the available resources in the scoreboard are equal to or greater than the resources required by the subgraph to be processed, then it can be determined that there are enough available resources in the scoreboard to support the configuration of the subgraph to be processed. Alternatively, if based on the resources required by the subgraph to be processed and the available resources in the scoreboard, it is determined that the available resources in the scoreboard are greater than the resources required by the subgraph to be processed, and the remaining available resources after subtracting the resources required by the subgraph to be processed from the available resources in the scoreboard are substantial, then it can be determined that there are enough available resources in the scoreboard to support the configuration of the subgraph to be processed.

[0064] The resources required for the subgraph to be processed can be determined based on the number of operators to be processed included in the subgraph. Based on this, the scheduler can determine the number of operators to be processed included in the subgraph, and determine the resources required for the subgraph. Optionally, the number of operators to be processed is equal to the amount of resources required for the subgraph.

[0065] The scoreboard can correspond to multiple registers, meaning it can record the data loading status of multiple registers. The scoreboard can record relevant data caching information at the granularity of a single operator. Therefore, if the entire scoreboard is viewed as a table, the index of this table represents the operator identifier ID of each operator, and each operator corresponds to a register recorded in the scoreboard, called an entry. In this embodiment, each operator to be processed in the subgraph corresponds to a register recorded in the scoreboard. Therefore, the number of operators to be processed in the subgraph can be equal to the number of registers required by the subgraph. Based on this, the number of operators to be processed in the subgraph and the number of free registers recorded in the scoreboard can be determined. Based on the number of operators to be processed and the number of free registers recorded in the scoreboard, it is checked whether there are enough free resources in the scoreboard to support the configuration of the subgraph.

[0066] Step S120: If it is found that there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed, then execute the subgraph configuration instruction to configure the subgraph to be processed in the scoreboard, wherein the subgraph configuration instruction is determined according to the data dependency relationship between the multiple operators to be processed included in the subgraph to be processed.

[0067] Optionally, the subgraph configuration instruction can be used to instruct the configuration of the subgraph to be processed in the scoreboard, wherein the subgraph configuration instruction can be used to instruct the configuration of each operator to be processed included in the subgraph to be processed in the scoreboard.

[0068] In this embodiment, if it is detected that there are enough free resources on the scoreboard to support the configuration of the subgraph to be processed, the subgraph configuration instruction can be executed, and the subgraph to be processed can be configured on the scoreboard. Alternatively, if it is detected that there are not enough free resources on the scoreboard to support the configuration of the subgraph to be processed, the subgraph configuration instruction can be temporarily not executed until it is detected that there are enough free resources on the scoreboard to support the configuration of the subgraph to be processed, at which point the subgraph configuration instruction can be executed, and the subgraph to be processed can be configured on the scoreboard.

[0069] In some implementations, the subgraph configuration instructions include operator configuration instructions (SCB OP config instructions) corresponding to each of the multiple operators to be processed within the subgraph to be processed. If sufficient free resources are found in the scoreboard to support the configuration of the subgraph to be processed, the operator configuration instructions corresponding to each of the multiple operators to be processed can be executed separately, and the configuration of the multiple operators to be processed can be performed separately in the corresponding registers of the scoreboard. Specifically, for each operator instruction within the subgraph, the compiler can generate the corresponding operator-level SCB OP config instruction based on data dependencies. The SCB OP config instruction describes the corresponding operator ID, the input buffer ID corresponding to each operator, the total capacity of the output buffer corresponding to each operator, etc. These parameters are filled into the corresponding registers of the scoreboard during configuration.

[0070] Step S130: Execute the second scoreboard check instruction to check whether the sub-graph to be processed has been configured in the scoreboard.

[0071] Optionally, the second scoreboard check instruction (second SCB CHECK instruction) can be used to determine whether the subgraph to be processed has been configured in the scoreboard before the first instruction of the subgraph to be processed is issued to the corresponding sub-computing engine, prior to the execution of the subgraph to be processed. That is, the second scoreboard check instruction can be used to ensure that the corresponding subgraph to be processed has been configured in the scoreboard. The scheduler may include a scoreboard configuration engine, which can execute the second scoreboard check instruction to check whether the subgraph to be processed has been configured in the scoreboard.

[0072] In this embodiment, after configuring the subgraph to be processed in the scoreboard, the second scoreboard check instruction can be executed to check whether the subgraph to be processed has been configured in the scoreboard. As one approach, the second scoreboard check instruction can be executed to check whether the multiple operators included in the subgraph to be processed have been configured in the scoreboard. If it is determined that all multiple operators included in the subgraph to be processed have been configured in the scoreboard, then the subgraph to be processed can be determined to be configured in the scoreboard; or, if it is determined that any one or more of the multiple operators included in the subgraph to be processed have not been configured in the scoreboard, then the subgraph to be processed can be determined to be not configured in the scoreboard.

[0073] Step S140: If it is found that the subgraph to be processed has been configured in the scoreboard, then the scheduling control is used to determine whether to allow the plurality of operators to be processed according to the configuration information of the subgraph to be processed recorded in the scoreboard. The configuration information of the subgraph to be processed includes the input buffer information, output buffer information and various counters related to the output buffer for each of the plurality of operators to be processed.

[0074] Optionally, if it is detected that the subgraph to be processed has been configured in the scoreboard, the configuration information of the subgraph to be processed recorded in the scoreboard can be obtained, and scheduling control can be used to determine whether to allow multiple pending operators based on the configuration information of the subgraph to be processed recorded in the scoreboard. The scheduling control to determine whether to allow multiple pending operators may include checking the data hazards and structural hazards of each of the multiple pending operators, and determining whether to allow the corresponding pending operator based on the detected data hazards and structural hazards. Alternatively, if it is detected that the subgraph to be processed has not been configured in the scoreboard, scheduling control can be temporarily suspended until it is detected that the subgraph to be processed has been configured in the scoreboard. Then, the configuration information of the subgraph to be processed recorded in the scoreboard can be obtained, and scheduling control can be used to determine whether to allow multiple pending operators based on the configuration information of the subgraph to be processed recorded in the scoreboard.

[0075] In some implementations, the configuration information of the subgraph to be processed includes input buffer information, output buffer information, and various counters related to the output buffer for each of the multiple operators included in the subgraph. Specifically, if the operator to be processed is the initial input operator in the AI ​​network, it can be assumed that the operator has no input buffer. Therefore, the corresponding input buffer information can be left blank in the register corresponding to the operator. Similarly, if the operator to be processed has no output buffer, the corresponding output buffer information and various counters related to the output buffer can be left blank in the register corresponding to the operator.

[0076] In this embodiment, the number of input buffers corresponding to each of the multiple operators to be processed can be 0, 1, or more. Therefore, the scoreboard can record the number of its corresponding input buffers in the corresponding register, as well as the input buffer identification information (input buffer ID) used to determine each input buffer. Thus, the input buffer information can include the number of input buffers and the input buffer identification information used to determine each input buffer.

[0077] Understandably, each register recorded on the scoreboard corresponds to an operator, and each operator can theoretically have an input buffer and an output buffer, which are independent of each other. For each register, this embodiment primarily aims to record information about the output buffer, including the amount of valid data and free space. However, for each operator, the information about its input buffer also needs to be maintained, as it may be necessary to check whether the amount of valid data in the input buffer meets a threshold later. Therefore, this embodiment provides an index ID (i.e., input buffer identification information) for the input buffer of the operator to be processed in a register corresponding to that operator.

[0078] It should be noted that a cache may be the input cache of one or more operators, and also the output cache of one or more other operators. For example, suppose there is a first operator, a second operator, and a target cache. When the first operator executes, it needs to read data from the target cache; in this case, the target cache can be considered the input cache of the first operator. Simultaneously, when the second operator executes, it needs to write data to the target cache; in this case, the target cache can be considered the output cache of the second operator. That is, the input cache of the first operator is also the output cache of the second operator. In this embodiment, the operator identifier information of an operator can generally be set to be consistent with the output cache identifier information of its corresponding output cache. Therefore, when the input cache of the first operator serves as the output cache of the second operator, the output cache identifier information of the second operator's output cache can be the operator identifier information of the second operator. Correspondingly, the input cache identifier information of the first operator's input cache can also be the operator identifier information of the second operator.

[0079] Furthermore, because in AI networks, the input buffer of each operator is typically the output buffer of other operators, when using a specific input buffer as an output buffer, the operator identifier information of the corresponding operator can be used as the index value. This index value reveals the corresponding register, which records detailed information about the input of that operator (corresponding to the output buffers of other operators).

[0080] In this embodiment, the number of output buffers corresponding to each of the multiple operators to be processed can be 0, 1, or more. Therefore, the scoreboard can record detailed information about each corresponding output buffer in its corresponding register. Furthermore, for each output buffer, the valid data in the output buffer may be consumed by one or more other operators; therefore, the detailed information for each output buffer may include, but is not limited to:

[0081] The number of consumers corresponding to each output buffer;

[0082] Total capacity of all output buffers;

[0083] The effective data volume and free space in each output buffer. The following combinations of information can provide equivalent information: cumulative consumption in the output buffer & cumulative generation in the output buffer & total capacity of the output buffer; remaining space in the output buffer & total capacity of the output buffer; effective data volume in the output buffer & total capacity of the output buffer;

[0084] When the output buffer has multiple consumers, each consumer needs to be recorded separately (e.g., the corresponding rcntr for each consumer, including the rcntr and psv_rcntr for each consumer). Because different consumers may consume data from the same output buffer at different speeds, slower-consuming consumers need to be considered when calculating free space. This avoids prematurely releasing space, which could prevent slower-consuming consumers from obtaining the correct data; for example... Figure 3 As shown, there are two different consumers: operator 1 and operator 2.

[0085] The amount of data to be consumed and generated by the issued instructions. When there are multiple consumption nodes, the amount of data to be consumed must be recorded separately for each node.

[0086] The effective data volume and free space within the output buffer can be determined by the data index rcntr that each consumer will read from the output buffer, the data index wcntr that each producer will write to the output buffer, the data index psv_rcntr that needs to be reserved in the output buffer for each consumer, and the free space psv_wcntr that needs to be reserved in the output buffer for each producer. Therefore, this output buffer information can include the number of output buffers, the total capacity of each output buffer, and the number of consumers corresponding to each output buffer. The various counters related to the output buffer may include the data index that each consumer will read from the output buffer next, the data index that each producer will write to the output buffer next, the data index that the output buffer needs to reserve for each consumer, and the free space that the output buffer needs to reserve for each producer. Optionally, the various counters related to the output may include four types of counters (first counter, second counter, third counter, and fourth counter), wherein the first counter corresponds to the data index that each consumer will read from the output buffer next, the second counter corresponds to the data index that each producer will write to the output buffer next, the third counter corresponds to the data index that the output buffer needs to reserve for each consumer, and the fourth counter corresponds to the free space that the output buffer needs to reserve for each producer.

[0087] Therefore, this embodiment uses a relatively concise counter value to calculate and determine the effective data volume and free space in the cache, and supports read and write operations by multiple consumers and multiple producers. The implementation is simple and does not consume excessive hardware resources. It is highly scalable and adaptable to different modes. Furthermore, using different types of counters can express the effective data volume, free space, and data volume that needs to be reserved in the cache, not only satisfying data dependency checks but also preventing the risk of over-issuance of the same data or space by different operators simultaneously.

[0088] Optionally, the configuration information of the subgraph to be processed may also include sub-engine ID, which is used to determine the sub-computing engine whose instruction corresponding to the operator to be processed is executed.

[0089] It is understandable that the definition of the registers in this embodiment is determined by software, which can be easily customized according to different algorithms or requirements. The related solutions have high compatibility and strong scalability.

[0090] An embodiment of this application provides an instruction processing method that first executes a first scoreboard check instruction to check whether there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed. If it is found that there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed, a subgraph configuration instruction is executed to configure the subgraph to be processed in the scoreboard. The subgraph configuration instruction is determined based on the data dependencies between the multiple operators to be processed included in the subgraph to be processed. Then, a second scoreboard check instruction is executed to check whether the subgraph to be processed has been configured in the scoreboard. If it is found that the subgraph to be processed has been configured in the scoreboard, the configuration of the subgraph to be processed is performed according to the configuration information recorded in the scoreboard. The information scheduling control determines whether to allow multiple pending operators. The configuration information of the pending subgraph includes the input buffer information, output buffer information, and various counters related to the output buffer for each pending operator. Based on the data dependencies between the operators in the subgraph, the subgraph is configured in the scoreboard, and the input buffer information, output buffer information, and various counters related to the output buffer for each operator are recorded in the scoreboard. This provides a concise and clear representation of the relevant data storage information, serving as a reference for whether to allow operators in the subgraph. This effectively resolves the data dependencies between operators and improves processing efficiency.

[0091] Please see Figure 4 , Figure 4 A flowchart illustrating an embodiment of the instruction processing method provided in this application is shown. This method is applied to a scheduler, and will be discussed below. Figure 4 The process shown will be described in detail. The instruction processing method may specifically include the following steps:

[0092] Step S210: Execute the first scoreboard check instruction to check whether there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed.

[0093] Step S220: If it is found that there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed, then execute the subgraph configuration instruction to configure the subgraph to be processed in the scoreboard, wherein the subgraph configuration instruction is determined according to the data dependency relationship between the multiple operators to be processed included in the subgraph to be processed.

[0094] For a detailed description of steps S210-S220, please refer to steps S110-S120, which will not be repeated here.

[0095] Step S230: If it is determined that the configuration of the subgraph to be processed has been completed in the scoreboard, then execute the first scoreboard update instruction to update the configuration quantity of the effective data cache space in the scoreboard.

[0096] The compiler can segment subgraphs according to the on-chip memory specifications of the overall AI processor. Before each subgraph enters the configuration phase, a corresponding scoreboard check instruction is inserted; after each subgraph is configured, a corresponding scoreboard update instruction is inserted; and before each subgraph enters the execution phase, a corresponding scoreboard check instruction is inserted. In this embodiment, for the segmented subgraphs to be processed, a first scoreboard check instruction can be inserted before the subgraph enters the configuration phase, a first scoreboard update instruction can be inserted after the subgraph is configured, and a second scoreboard check instruction can be inserted before the subgraph enters the execution phase.

[0097] Optionally, the first scoreboard update instruction (first SCB UDPATE instruction) can be used to indicate the configured amount of valid data cache space in the updated scoreboard. Of course, the first scoreboard update instruction can also further instruct that instructions for the operators to be processed contained in the corresponding subgraph to be processed can be sent to the corresponding sub-computation engine for execution.

[0098] In this embodiment, after configuring the subgraph to be processed in the scoreboard, the first scoreboard update instruction can be executed to update the configuration quantity of the effective data cache space in the scoreboard. This ensures the timely update of the configuration quantity recorded in the scoreboard, avoids the repeated application of data cache space in the scoreboard, and ensures the accuracy of the scoreboard records.

[0099] The scheduler may include a scoreboard configuration engine, which can execute a first scoreboard update instruction to update the configured number of valid data cache spaces in the scoreboard.

[0100] Step S240: Execute the second scoreboard check instruction to check whether the sub-graph to be processed has been configured in the scoreboard.

[0101] It is understood that this embodiment involves two checks to verify whether the subgraph to be processed has been configured in the scoreboard. The first check occurs during the configuration phase of the subgraph. Performing this check during the configuration phase ensures that the number of configurations recorded in the scoreboard is updated promptly when the subgraph is configured, preventing duplicate allocation of data cache space in the scoreboard and ensuring the accuracy of the scoreboard records. The second check occurs before the execution of the subgraph. Performing this check before execution ensures that the instructions corresponding to the operators contained within the subgraph are executed only after the subgraph has been configured, guaranteeing that the instructions corresponding to the operators can be executed successfully.

[0102] Step S250: If it is found that the subgraph to be processed has been configured in the scoreboard, then the scheduling control is used to determine whether to allow the plurality of operators to be processed according to the configuration information of the subgraph to be processed recorded in the scoreboard. The configuration information of the subgraph to be processed includes the input buffer information, output buffer information and various counters related to the output buffer for each of the plurality of operators to be processed.

[0103] For a detailed description of steps S240-S250, please refer to steps S130-S140, which will not be repeated here.

[0104] An embodiment of this application provides an instruction processing method that, compared to... Figure 2 The instruction processing method shown in this embodiment also executes a first scoreboard update instruction before executing the second scoreboard check instruction. This instruction is used to update the number of configurations in the effective data cache space in the scoreboard when it is determined that the configuration of the subgraph to be processed has been completed in the scoreboard. This can ensure the timely update of the number of configurations recorded in the scoreboard and ensure the accuracy of the scoreboard records.

[0105] Please see Figure 5 , Figure 5 A flowchart illustrating an embodiment of the instruction processing method provided in this application is shown. This method is applied to a scheduler, and will be discussed below. Figure 5 The process shown will be described in detail. The instruction processing method may specifically include the following steps:

[0106] Step S310: Execute the first scoreboard check instruction to check whether there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed.

[0107] Step S320: If it is found that there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed, then execute the subgraph configuration instruction to configure the subgraph to be processed in the scoreboard, wherein the subgraph configuration instruction is determined according to the data dependency relationship between the multiple operators to be processed included in the subgraph to be processed.

[0108] Step S330: Execute the second scoreboard check instruction to check whether the sub-graph to be processed has been configured in the scoreboard.

[0109] Step S340: If it is found that the subgraph to be processed has been configured in the scoreboard, then the scheduling control is used to determine whether to allow the plurality of operators to be processed according to the configuration information of the subgraph to be processed recorded in the scoreboard. The configuration information of the subgraph to be processed includes the input buffer information, output buffer information and various counters related to the output buffer for each of the plurality of operators to be processed.

[0110] For a detailed description of steps S310-S340, please refer to steps S110-S140, which will not be repeated here.

[0111] Step S350: If it is determined that the instructions corresponding to the plurality of pending operators have been executed, then execute the second scoreboard update instruction to notify that the resources in the scoreboard corresponding to the pending subgraph have been released.

[0112] The compiler can segment subgraphs according to the on-chip memory specifications of the overall AI processor. Before each subgraph enters the configuration phase, a corresponding scoreboard check instruction is inserted; before each subgraph enters the execution phase, a corresponding scoreboard check instruction is inserted; and after the execution of instructions corresponding to the multiple operators included in the subgraph to be processed, a corresponding scoreboard update instruction is inserted. In this embodiment, for the segmented subgraphs to be processed, a first scoreboard check instruction can be inserted before the subgraph enters the configuration phase, a second scoreboard check instruction can be inserted before the subgraph enters the execution phase, and a second scoreboard update instruction can be inserted after the execution of instructions corresponding to the multiple operators included in the subgraph.

[0113] Optionally, the second scoreboard update instruction (second SCB UDPATE instruction) can be used to notify that the relevant scoreboard resources have been released. Therefore, in this embodiment, after determining that the instructions corresponding to the multiple operators to be processed included in the subgraph to be processed have been executed, the second scoreboard update instruction can be executed to notify that the resources in the scoreboard corresponding to the subgraph to be processed have been released, so that other subgraphs can apply for the free resources in the scoreboard for configuration operations.

[0114] The scheduler may include a scoreboard configuration engine, which can execute a second scoreboard update instruction to notify that the resources corresponding to the subgraph to be processed in the scoreboard are released.

[0115] An embodiment of this application provides an instruction processing method that, compared to... Figure 2The instruction processing method shown in this embodiment further includes executing a second scoreboard update instruction after scheduling control to allow multiple pending operators to proceed. This instruction is used to release the resources corresponding to the pending subgraph in the scoreboard after the pending subgraph is executed, ensuring the timely update of idle resources in the scoreboard for timely use by subsequent subgraphs and improving processing efficiency.

[0116] Of course, this instruction sequence can also simultaneously include a first scoreboard check instruction, a subgraph configuration instruction, a first scoreboard update instruction, a second scoreboard check instruction, a subgraph to be processed, and a second scoreboard update instruction. Based on this, it can be performed as follows: Figure 6 The execution process shown will not be elaborated further here.

[0117] Please see Figure 7 , Figure 7 A flowchart illustrating an embodiment of the instruction processing method provided in this application is shown. This method is applied to a scheduler, and will be discussed below. Figure 7 The process shown will be described in detail. The instruction processing method may specifically include the following steps:

[0118] Step S410: Execute the first scoreboard check instruction to check whether the idle resources in the scoreboard are sufficient to support the synchronous processing of at least two subgraphs, wherein the at least two subgraphs include the subgraph to be processed, and the synchronous processing of at least two subgraphs includes the configuration of the subgraph to be processed and the execution of instructions corresponding to operators included in other subgraphs of the at least two subgraphs.

[0119] In order to simultaneously complete the configuration and instruction execution of different subgraphs, the scoreboard needs to reserve a significant amount of idle resources (e.g., reserved resources to support the information configuration of at least two subgraphs) to enable parallel processing of subgraph information configuration and instruction execution, thereby improving performance. Therefore, in this embodiment, it can be checked whether the idle resources in the scoreboard are sufficient to support the synchronous processing of at least two subgraphs, where the at least two subgraphs include the subgraph to be processed, and the synchronous processing of the at least two subgraphs includes configuring the subgraph to be processed and executing the instructions corresponding to the operators included in other subgraphs within the at least two subgraphs.

[0120] In some implementations, the scheduler can determine the resources required for configuring the subgraph to be processed, the resources required for executing instructions corresponding to operators in other subgraphs, and the available resources in the scoreboard. Based on the resources required for configuring the subgraph to be processed, the resources required for executing instructions corresponding to operators in other subgraphs, and the available resources in the scoreboard, the scheduler checks whether there are enough available resources in the scoreboard to support the synchronous processing of at least two subgraphs. Optionally, if based on the resources required for configuring the subgraph to be processed, the resources required for executing instructions corresponding to operators in other subgraphs, and the available resources in the scoreboard, it is determined that the available resources in the scoreboard are equal to or greater than the sum of the resources required for configuring the subgraph to be processed and the resources required for executing instructions corresponding to operators in other subgraphs, then it can be determined that there are enough available resources in the scoreboard to support the synchronous processing of at least two subgraphs.

[0121] Step S420: If it is found that the idle resources in the scoreboard are sufficient to support the synchronous processing of at least two subgraphs, then it is determined that there are enough idle resources in the scoreboard to support the configuration of the subgraph to be processed, and the corresponding idle resources are allocated for the configuration of the subgraph to be processed.

[0122] In this embodiment, if it is found that the scoreboard has enough idle resources to support the synchronous processing of at least two subgraphs, it can be determined that there are enough resources in the scoreboard to support the configuration of the subgraph to be processed, and the corresponding idle resources are allocated for the configuration of the subgraph to be processed. Based on this, it is ensured that the information configuration and instruction execution of the same subgraph are strictly executed in sequence, and the configuration and instruction execution of different subgraphs are parallelized, thereby maximizing the overall program execution efficiency.

[0123] like Figure 8 As shown, Figure 8 The code assumes two independent scoreboard resources, each capable of supporting the configuration and execution of two instruction sequence segments. That is, as follows: Figure 8 As shown, sub-graph 0 includes the execution phase of the instruction corresponding to the operator and the sub-graph. Figure 1 The configuration phase can be processed synchronously. However, if there is only one independent scoreboard resource, then when the instructions corresponding to the operators included in subgraph 0 have not been executed, it indicates that there are no idle scoreboard resources available for subgraph configuration. Figure 2 configuration, sub Figure 2 The configuration fails to start; similarly, the sub-configuration fails to start. Figure 2 The execution of the corresponding command cannot be started until the configuration is complete.

[0124] like Figure 9As shown, when the configuration and instruction execution of subgraphs are not parallelized, each subgraph needs to proceed sequentially from configuration to execution of the previous subgraph before proceeding to configuration and execution of the next subgraph, resulting in low processing efficiency. When the configuration and instruction execution of subgraphs are parallelized, the information configuration and instruction execution within the same subgraph are strictly executed in sequence, while the configuration and instruction execution of different subgraphs are processed synchronously, resulting in higher processing efficiency.

[0125] Step S430: If it is found that there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed, then execute the subgraph configuration instruction and configure the subgraph to be processed in the scoreboard.

[0126] Step S440: Execute the second scoreboard check instruction to check whether the sub-graph to be processed has been configured in the scoreboard.

[0127] Step S450: If it is found that the subgraph to be processed has been configured in the scoreboard, then the scheduling control is used to determine whether to allow the plurality of operators to be processed according to the configuration information of the subgraph to be processed recorded in the scoreboard. The configuration information of the subgraph to be processed includes the input buffer information, output buffer information and various counters related to the output buffer for each of the plurality of operators to be processed.

[0128] For a detailed description of steps S430-S450, please refer to steps S120-S140, which will not be repeated here.

[0129] An embodiment of this application provides an instruction processing method that, compared to... Figure 2 The instruction processing method shown in this embodiment checks whether the idle resources in the scoreboard are sufficient to support the synchronous processing of at least two subgraphs. The at least two subgraphs include a subgraph to be processed. The synchronous processing of the at least two subgraphs includes the configuration of the subgraph to be processed and the execution of instructions corresponding to operators included in other subgraphs in the at least two subgraphs. If it is found that the idle resources in the scoreboard are sufficient to support the synchronous processing of at least two subgraphs, it is determined that there are enough idle resources in the scoreboard to support the configuration of the subgraph to be processed, and the corresponding idle resources are allocated for the configuration of the subgraph to be processed. In this way, by reserving more idle resources in the scoreboard, the information configuration and instruction execution of the subgraphs can be parallelized, thereby improving performance.

[0130] Please see Figure 10 , Figure 10 A flowchart illustrating an embodiment of the instruction processing method provided in this application is shown. This method is applied to a scheduler. In this embodiment, the scoreboard corresponds to multiple registers, and the subgraph configuration instructions include operator configuration instructions corresponding to each of the multiple operators to be processed. The following will focus on... Figure 10The process shown will be described in detail. The instruction processing method may specifically include the following steps:

[0131] Step S510: Execute the first scoreboard check instruction to check whether there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed.

[0132] For a detailed description of step S510, please refer to step S110, which will not be repeated here.

[0133] Step S520: If it is found that there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed, then execute the operator configuration instructions corresponding to each of the plurality of operators to be processed, and allocate one of the plurality of registers to each operator to be processed.

[0134] In this embodiment, the scoreboard corresponds to multiple registers, that is, the scoreboard is used to record the data loading information of multiple registers, and the subgraph configuration instructions include operator configuration instructions corresponding to each of the multiple operators to be processed.

[0135] During the configuration phase of the subgraph to be processed, the scheduler can execute the operator configuration instructions corresponding to each of the multiple operators to be processed, and allocate one of the multiple registers to each operator to be processed. Optionally, a register (containing the corresponding entry ID, where the entry ID is the operator identification information of the operator to be processed in the entire scoreboard) can be allocated to each operator to be processed according to the corresponding instruction information.

[0136] Step S530: In each of the allocated registers, write the input buffer information and the total capacity of the output buffer corresponding to the operator to be processed.

[0137] In this embodiment, when one of the multiple registers is allocated to each operator to be processed, the input buffer information and the total capacity of the output buffer corresponding to the operator to be processed can be written into each allocated register.

[0138] Optionally, the engine identification information corresponding to the operator to be processed can also be written into each allocated register, which is not limited here.

[0139] In AI processors, data processing granularity is often based on tensors or vectors. Therefore, data hazard detection needs to be extended from traditional register granularity to the broader memory domain. In AI, a basic data cache unit is often defined based on different granularities such as layer / line / tile / cube. In this embodiment, a minimum data cache can be defined based on tile granularity. The size of the tile is determined by the compiler and can represent different granularities such as layer / line / cube under specific configurations. Correspondingly, in this embodiment, the data recorded in the register can be in the unit of tile. For example, if the threshold for input data in an operator checking instruction is defined as 2, then it means that there must be 2 tiles of data before subsequent execution can proceed.

[0140] Step S540: Execute the second scoreboard check instruction to check whether the sub-graph to be processed has been configured in the scoreboard.

[0141] For a detailed description of step S540, please refer to step S130, which will not be repeated here.

[0142] Step S550: If it is found that the subgraph to be processed has been configured in the scoreboard, then a target operator to be processed is determined from the plurality of operators to be processed, wherein the target operator to be processed is any one of the plurality of operators to be processed, an operator check instruction is inserted before the target operator to be processed, the operator check instruction carries the effective amount of data required for the execution of the target operator to be processed, and / or the free space required for the data output by the target operator to be processed.

[0143] During the execution phase of the instructions corresponding to the operators, before each execution instruction is sent to the corresponding sub-computing engine, the scheduler queries the scoreboard based on the preceding operator check instruction. The operator check instruction carries a threshold for the effective data volume and / or free space required to execute the operator. If the effective data volume and / or free space of the corresponding data cache retrieved from the scoreboard meet this threshold, the current instruction can be sent for execution; otherwise, it must wait for the scoreboard to update the corresponding value to meet the requirements.

[0144] The execution of instructions corresponding to multiple pending operators included in the pending subgraph can be determined by identifying the target pending operator from among the multiple pending operators. This target pending operator can be any one of the multiple pending operators. For this target pending operator, an operator check instruction can be inserted before it. This operator check instruction carries the effective amount of data required for the execution of the target pending operator and / or the free space required for the output data of the target pending operator, in order to check whether to allow the target pending operator to proceed.

[0145] Optionally, the compiler can generate corresponding operator checking instructions based on the target operator to be processed. Furthermore, the compiler can define the checking instructions; specifically, the operator checking instructions can be defined as indicating that the target operator to be processed will only be allowed to proceed when the amount of valid data in the target input buffer corresponding to the target operator is sufficient for the execution of the target operator, and / or the free space in the target output buffer corresponding to the target operator is sufficient for the output data of the target operator.

[0146] It is understandable that for a target operator to be processed, if it has corresponding input data, the prerequisite for its execution includes that the corresponding operation data is ready and can be obtained. As an example, assuming the target operator requires 32 bits of valid data to execute, then this input data threshold can be defined as 32 bits. That is, the amount of valid data in the target input buffer corresponding to the target operator can be defined to be at least 32 bits.

[0147] It is understandable that for a target operator to be processed, if it has corresponding output data, a prerequisite for its execution includes that the storage space for the corresponding data output must have sufficient free space to accommodate the output data of the target operator. For example, assuming the execution of the target operator will produce 64 bits of output data, then the threshold for the storage space of this output data can be defined as 64 bits. That is, it can be defined that the free space in the target output buffer corresponding to the target operator must be at least 64 bits.

[0148] In this context, it's understandable that for a target operator to be processed, if it has corresponding input and output data, the prerequisites for its execution include that the corresponding operation data is ready and can be obtained, and that the storage space for the corresponding data output has sufficient free space to accommodate the output data of the target operator. For example, assuming the target operator requires 32 bits of valid data to execute and will produce 64 bits of output data, then the input data threshold can be defined as 32 bits, and the storage space threshold for the output data can be defined as 64 bits. That is, it can be defined that the amount of valid data in the target input buffer corresponding to the target operator must be at least 32 bits, and that the free space in the target output buffer corresponding to the target operator must be at least 64 bits.

[0149] Therefore, in this embodiment, when a target operator to be processed is determined, the compiler can determine whether the target operator corresponds to input data and whether it corresponds to output data. Specifically, if it is determined that the target operator only corresponds to input data and not output data, a check instruction can be defined to instruct that the target operator is executed only if the amount of valid data in the target input buffer corresponding to the target operator is sufficient for its execution. Similarly, if it is determined that the target operator only corresponds to output data and not input data, a check instruction can be defined to instruct that the target operator is executed only if the available space in the target output buffer corresponding to the target operator is sufficient for its output data. If it is determined that the target operator to be processed has corresponding input data and corresponding output data, a check instruction can be defined to indicate that the target operator to be processed will only be executed if the amount of valid data in the target input buffer corresponding to the target operator to be processed reaches the amount of valid data required for the execution of the target operator to be processed, and the free space in the target output buffer corresponding to the target operator to be processed reaches the free space required for the output data of the target operator to be processed.

[0150] In this process, the compiler, after determining the target operator to be processed and generating the operator checking instruction, can insert the operator checking instruction before the target operator to be processed. By inserting the operator checking instruction before the target operator to be processed, the hardware can execute the operator checking instruction before executing the target operator to be processed. This allows the target operator to be processed to be executed according to the instructions of the operator checking instruction; that is, the target operator to be processed is only executed if the amount of valid data in the target input buffer corresponding to the target operator is sufficient for the target operator to execute, and / or if the free space in the target output buffer corresponding to the target operator is sufficient for the data output by the target operator.

[0151] Step S560: Execute the operator check instruction, query the scoreboard to determine whether the parameter information in the target cache corresponding to the target operator meets the specified parameter information, wherein the specified parameter information includes the amount of valid data in the target input cache corresponding to the target operator reaching the amount of valid data required for the execution of the target operator, and / or the free space in the target output cache corresponding to the target operator reaching the free space required for the output data of the target operator.

[0152] In this embodiment, the scheduler can be responsible for executing operator check instructions and determining whether to allow the target operator to proceed based on the execution status of the operator check instructions. That is, the scheduler executes the operator check instructions, queries the scoreboard, and determines whether the parameter information in the target cache corresponding to the target operator meets the specified parameter information. The parameter information in the target cache may include the effective data volume in the corresponding target input cache and / or the free space in the corresponding target output cache.

[0153] In some implementations, the effective data volume in the target input buffer can be obtained as follows: Based on the operator identifier information of the target operator to be processed, the input identifier information of the target input buffer corresponding to the target operator to be processed is found, where the target input buffer is another output buffer corresponding to other operators, and other operators are one or more operators to be processed other than the target operator to be processed; based on the input identifier information of the target input buffer, the first register corresponding to the other operator is determined; based on the configuration information recorded in the first register, the effective data volume in the target input buffer corresponding to the target operator to be processed is calculated. That is, based on the operator ID of the target operator to be processed, its corresponding target input buffer ID can be found, and then the corresponding entry can be found through the target input buffer ID. The effective data volume visible to the operator to be processed can be calculated using the counter recorded on the corresponding entry.

[0154] As one possible approach, the configuration information recorded in the first register includes the data indexes that other output caches need to reserve for each consumer, and the data indexes that each producer of the other output caches will subsequently write into the other output caches. Based on this, calculating the effective data volume in the target input cache corresponding to the target operator to be processed, according to the configuration information recorded in the first register, may include: calculating the effective data volume in the target input cache corresponding to the target operator to be processed, based on the data indexes that other output caches need to reserve for each consumer and the data indexes that producers of the other output caches will subsequently write into the other output caches. For example, if the data index that other output caches need to reserve for each consumer is psv_rcntr, and the data index that each producer of the other output caches will subsequently write into the other output cache is wcntr, then the effective data volume in the target input cache is wcntr - psv_rcntr.

[0155] In some implementations, the free space in the target output buffer can be obtained by: finding the second register corresponding to the target operator based on its operator identifier information; and calculating the free space in the target output buffer corresponding to the target operator based on the configuration information recorded in the second register. That is, the free space available for the operator can be calculated directly by querying the corresponding entry in the scoreboard based on the operator ID of the target operator, and then using the counter recorded in the corresponding entry.

[0156] As one feasible approach, the configuration information recorded in the second register includes the total capacity of the target output buffer, the data index that each consumer will read from the target output buffer next, and the free space that the target output buffer needs to reserve for each producer. Based on this, calculating the free space in the target output buffer corresponding to the target operator to be processed, according to the configuration information recorded in the second register, can include: calculating the free space in the target output buffer corresponding to the target operator to be processed, based on the total capacity of the target output buffer, the data index that each consumer will read from the target output buffer next, and the free space that the target output buffer needs to reserve for the producer. As an example, if the total capacity of the target output buffer is Total Space, the data index rcntr that each consumer will read from the target output buffer next, and the free space psv_wcntr that the target output buffer needs to reserve for the producer, then the free space in the target output buffer is TotalSpace - (psv_wcntr - rcntr).

[0157] Therefore, it is understandable that this embodiment can use different counters to calculate and statistically analyze the effective data volume, free space, reserved data volume, and reserved free space of a single cache. To clearly illustrate the relevant technical solution, a specific example is provided below. In this example, for all data storage spaces in a specific subgraph, it is limited to a maximum of two different operators reading data from the space, referred to as consumer0 and consumer1, and a maximum of one operator writing data to this data space, defined as the producer. Figure 11 As shown, the corresponding scoreboard registers define a total of 6 different counters: rcntr0, rcntr1, wcntr, psv_rcntr0, psv_rcntr1, and psv_wcntr.

[0158] Here, rcntr0 and rcntr1 are the indices of the data that consumer0 and consumer1 will read next, respectively. This index is not necessarily equivalent to a specific address; it can be a data sequence number calculated using a custom data unit.

[0159] Here, wcntr is the index of the data that the producer will write next.

[0160] Here, psv_rcntr0 and psv_rcntr1 are the data indices that consumer0 and consumer1 need to retain, respectively. psv_rcntr-rcntr indicates that the corresponding data will be used as input data by subsequent operators. Once the relevant data is used, it will be considered invalid until the corresponding operator has finished executing, thereby preventing the same storage space data from being promised to other operators again as input data.

[0161] Here, psv_wcntr is the free space that the producer needs to reserve. psv_wcntr-wcntr means that the relevant free space has been reserved by the producer. This part of the free space is the space for storing the output data after the producer's relevant operators have been executed, and it cannot be used by other operators to write data.

[0162] Combining the above definitions, and referring to... Figure 12As can be seen, for consumer0, the corresponding valid data is wcntr-psv_rcntr0, and for consumer1, the valid data is wcntr-psv_rcntr1. The reserved space is psv_wcntr-min(rcntr0, rcntr1), and the free space is: Total Space-(psv_wcntr-min(rcntr0, rcntr1)).

[0163] Please refer to Figure 13 And Table 1, Figure 13 Table 1, based on the functional parameter definitions of the scoreboard above, provides a specific subgraph and its register information within the scoreboard. This use case includes: a subgraph and its corresponding operator graph, the register definitions of the scoreboard, and the corresponding information of each operator within the scoreboard.

[0164] Table 1

[0165]

[0166] Among them, Figure 13 In Table 1, ATO 20 is an operator numbered 20. This operator has two input buffers: 18 and 19, and one output buffer: 20. The topology between the corresponding operators 21, 22, 25, and 26 is as follows: Figure 13 As shown in Table 1, for this specific subgraph, the configuration information recorded in the corresponding registers on the scoreboard is as follows. Taking operator 20 as an example:

[0167] arg0 describes that it has 2 inputs and 1 output.

[0168] arg1 describes the IDs of the operators corresponding to its input buffer as 18 and 19. Since a buffer may have multiple consumers, here ridx indicates that operator 20 is the 0th consumer.

[0169] arg3 describes the total capacity of the current output buffer, the number of consumers, and provides the initial values ​​of the corresponding counters (i.e., the initial values ​​of counters in arg4 and arg5).

[0170] arg4 and arg5 give the values ​​of the corresponding counter.

[0171] In particular, considering that for more complex computational flow graphs, such as an operator that may correspond to multiple input buffers and multiple output buffers, the recording format of the input buffer information corresponding to the input buffers and the output buffer information corresponding to the output buffers in the scoreboard can be expanded, for example, but not limited to other formats such as linked lists.

[0172] Step S570: If it is determined that the parameter information in the target cache corresponding to the target operator to be processed meets the specified parameter information, then the target operator to be processed is allowed to proceed.

[0173] If it is determined that the amount of valid data in the target input buffer corresponding to the target operator reaches the amount of valid data required for the execution of the target operator, and / or the amount of free space in the target output buffer corresponding to the target operator reaches the amount of free space required for the output data of the target operator, then the target operator can be allowed to proceed.

[0174] In some implementations, when the target operator has corresponding input data, the operator check instruction is defined as including input buffer identification information and the amount of valid data required for the execution of the target operator. Based on this, when the scheduler executes the operator check instruction, it queries the scoreboard to determine the amount of valid data in the target input buffer corresponding to the input buffer identification information. If the amount of valid data in the target input buffer reaches the amount of valid data required for the execution of the target operator, the scheduler allows the target operator to proceed.

[0175] In some implementations, when the target operator has output data, the operator check instruction is defined as including the output buffer identifier information and the available space required for the output data of the target operator. Based on this, when the scheduler executes the operator check instruction, it queries the scoreboard to determine the available space of the target output buffer corresponding to the output buffer identifier information. If the available space of the target output buffer reaches the available space required for the output data of the target operator, the scheduler allows the target operator to proceed.

[0176] In some implementations, when the target operator has both input and output data, the operator check instruction is defined as including input buffer identification information and the amount of valid data required for the execution of the target operator, as well as the free space required for the output buffer identification information and the output data of the target operator. Based on this, when executing the operator check instruction, the scheduler queries the scoreboard and then determines whether the amount of valid data in the target input buffer corresponding to the input buffer identification information meets the amount of valid data required for the execution of the target operator, and whether the free space in the target output buffer corresponding to the output buffer identification information meets the free space required for the output data of the target operator. If the amount of valid data in the target input buffer meets the amount of valid data required for the execution of the target operator, and the free space in the target output buffer meets the free space required for the output data of the target operator, then the scheduler allows the target operator to proceed.

[0177] To achieve higher pipeline utilization efficiency, the AI ​​processor may allow another function instruction to be issued before the previous operator has completed, thereby achieving out-of-order execution of instructions. In this embodiment, to ensure that corresponding resources (valid data or spare space) are not repeatedly allocated to multiple operators, in addition to checking whether the resource quantity reaches a preset threshold, a mechanism for reserving and recording resources can be set up to ensure that the same data or space is not allocated to multiple operators (this error is called data or space over-allocation).

[0178] In some implementations, the reservation mechanism can be implemented using separate data reservation instructions, such as using corresponding data reservation instructions to indicate the amount of data or space that the hardware needs to reserve. In this embodiment, when the target operator instruction to be processed is determined, the compiler can generate operator check instructions and data reservation instructions based on the target operator to be processed. When the target operator to be processed, the operator check instructions, and the data reservation instructions are determined, the operator check instructions can be inserted before the target operator to be processed, and the data reservation instructions can be inserted between the operator check instructions and the target operator to be processed.

[0179] Understandably, since the data reservation instruction is inserted between the operator check instruction and the target operator to be processed, the hardware can execute the data reservation instruction after the check instruction is completed and before the target operator to be processed is issued, so as to reserve the corresponding resources.

[0180] When generating the data reservation instruction, the compiler can define the data reservation instruction to indicate the data index psv_rcntr that needs to be reserved for each consumer and the spare space psv_wcntr that needs to be reserved for the producer, which are recorded in the corresponding register in the scoreboard, to indicate the reservation of relevant data and spare space.

[0181] In this embodiment, the scheduler executes operator check instructions. Upon completion of these instructions, the scheduler updates the scoreboard to reserve relevant resources in the cache for subsequent execution of target operators. This prevents the risk of invalid data being illegally overwritten or insufficient free space due to over-issuance of resources. Therefore, if the operator check instructions are confirmed to be complete, before issuing subsequent target operators, the scheduler can execute data reservation instructions. Based on these instructions and the operator identifier of the target operator, the scheduler updates the data index recorded in the scoreboard that needs to be reserved for each consumer, as well as the free space that needs to be reserved for the producer.

[0182] Of course, the scheduler can reserve data differently according to the data reservation instruction based on the different ways that the target operator to be processed has input data and / or output data, which will not be elaborated here.

[0183] In some implementations, when a target operator to be processed is determined, the compiler can generate a corresponding operator update instruction based on that target operator. Furthermore, the compiler can define the operator update instruction; specifically, it can be defined as an instruction that, after the execution of the instruction corresponding to the target operator is completed, the data index rcntr to be read by each consumer and the data index wcntr to be written by each producer recorded in the scoreboard should be updated according to the operator identifier information of the target operator. Optionally, the operator update instruction can carry the data index to be read by each consumer and the data index to be written by each producer. Based on this, the relevant values ​​in the scoreboard can be updated according to the operator identifier information of the target operator.

[0184] It is understandable that, since the operator update instruction is inserted after the target operator to be processed, the hardware can execute the operator update instruction after executing the instructions corresponding to the target operator to be processed, that is, update the relevant value in the scoreboard.

[0185] Of course, the scheduler can update data differently based on the operator update instruction, depending on whether the target operator has input data and / or output data. This will not be elaborated further here.

[0186] Please see Figure 14 , Figure 14 This illustrates a schematic diagram of the instruction execution flow corresponding to the operators included in a subgraph, such as... Figure 14 As shown, for the operators in this subgraph, operator check instructions, data reservation instructions, and operator update instructions can be inserted to realize data hazard checks, data reservations, and scoreboard updates for each operator in the subgraph. The specific process will not be elaborated here.

[0187] An embodiment of this application provides an instruction processing method that, compared to... Figure 2 The instruction processing method shown in this embodiment further involves determining a target operator from multiple operators when the subgraph to be processed is configured in the scoreboard. The target operator is any one of the multiple operators. An operator check instruction is inserted before the target operator, carrying the effective data volume required for the target operator's execution and / or the free space required for the target operator's output data. The operator check instruction is executed, and the scoreboard is queried to determine whether the parameter information in the target cache corresponding to the target operator meets the specified parameter information. The specified parameter information includes that the effective data volume in the target input cache corresponding to the target operator meets the effective data volume required for the target operator's execution, and / or that the free space in the target output cache corresponding to the target operator meets the free space required for the target operator's output data. If the parameter information in the target cache corresponding to the target operator meets the specified parameter information, the target operator is allowed to proceed. This method can resolve data dependencies between instructions and improve processing efficiency.

[0188] In some embodiments, one example of this application provides an instruction processing device applied to the scheduler described above. The instruction processing device includes: a first scoreboard check instruction execution module, a subgraph configuration instruction execution module, a second scoreboard check instruction execution module, and a subgraph execution module to be processed, wherein:

[0189] The first scoreboard check instruction execution module is used to execute the first scoreboard check instruction to check whether there are enough free resources on the scoreboard to support the configuration of the subgraph to be processed.

[0190] Furthermore, the first scoreboard check instruction execution module includes: a first scoreboard check instruction execution submodule and an idle resource allocation submodule, wherein:

[0191] The first scoreboard check instruction execution submodule is used to execute the first scoreboard check instruction to check whether the idle resources in the scoreboard are sufficient to support the synchronous processing of at least two subgraphs, wherein the at least two subgraphs include the subgraph to be processed, and the synchronous processing of the at least two subgraphs includes the configuration of the subgraph to be processed and the execution of instructions corresponding to operators included in other subgraphs of the at least two subgraphs.

[0192] The idle resource allocation submodule is used to determine if there are enough idle resources in the scoreboard to support the configuration of the subgraph to be processed if it is found that there are enough idle resources in the scoreboard to support the synchronous processing of at least two subgraphs, and to allocate the corresponding idle resources to configure the subgraph to be processed.

[0193] The subgraph configuration instruction execution module is used to execute a subgraph configuration instruction on the scoreboard if it is found that there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed. The subgraph configuration instruction is determined based on the data dependency relationship between multiple operators to be processed included in the subgraph to be processed.

[0194] Furthermore, the scoreboard corresponds to multiple registers, the subgraph configuration instructions include operator configuration instructions corresponding to each of the multiple operators to be processed, and the subgraph configuration instruction execution module includes: an operator configuration instruction execution submodule and an information writing submodule, wherein:

[0195] The operator configuration instruction execution submodule is used to execute the operator configuration instructions corresponding to each of the plurality of operators to be processed, and to allocate one of the plurality of registers to each of the operators to be processed.

[0196] The information writing submodule is used to write the input buffer information and the total capacity of the output buffer corresponding to the operator to be processed into each of the allocated registers.

[0197] The second scoreboard check instruction execution module is used to execute the second scoreboard check instruction to check whether the sub-graph to be processed has been configured in the scoreboard.

[0198] The subgraph execution module is used to, if it is found that the subgraph to be processed has been configured in the scoreboard, schedule and control whether to allow the plurality of operators to be processed according to the configuration information of the subgraph to be processed recorded in the scoreboard. The configuration information of the subgraph to be processed includes input buffer information, output buffer information and various counters related to the output buffer for each of the plurality of operators to be processed.

[0199] Further, the subgraph execution module includes: a target operator determination submodule, an operator check instruction execution submodule, and a target operator release submodule, wherein:

[0200] The target operator determination submodule is used to determine the target operator from the plurality of operators to be processed, wherein the target operator is any one of the plurality of operators to be processed, and an operator check instruction is inserted before the target operator, the operator check instruction carrying the effective amount of data required for the execution of the target operator and / or the free space required for the output data of the target operator.

[0201] The operator check instruction execution submodule is used to execute the operator check instruction and query the scoreboard to determine whether the parameter information in the target cache corresponding to the target operator to be processed meets the specified parameter information. The specified parameter information includes the amount of effective data in the target input cache corresponding to the target operator to be processed reaching the amount of effective data required for the execution of the target operator to be processed, and / or the free space in the target output cache corresponding to the target operator to be processed reaching the free space required for the output data of the target operator to be processed.

[0202] The target operator release submodule is used to release the target operator if it is determined that the parameter information in the target cache corresponding to the target operator meets the specified parameter information.

[0203] Furthermore, the subgraph execution module to be processed includes: an input identifier information lookup submodule, a first register determination submodule, and an effective data volume calculation submodule, wherein:

[0204] The input identifier information lookup submodule is used to look up the input identifier information of the target input cache corresponding to the target operator based on the operator identifier information of the target operator to be processed, wherein the target input cache is another output cache corresponding to other operators.

[0205] The first register determination submodule is used to determine the first register corresponding to the other operator based on the input identification information of the target input buffer, wherein the other operator is one or more operators to be processed other than the target operator to be processed.

[0206] The effective data volume calculation submodule is used to calculate the effective data volume in the target input buffer corresponding to the target operator to be processed based on the configuration information recorded in the first register.

[0207] Furthermore, the configuration information recorded in the first register includes the data indexes that the other output caches need to retain for each consumer, and the data indexes that each producer of the other output caches will write to the other output caches next. The effective data volume calculation submodule includes: an effective data volume calculation unit, wherein:

[0208] Based on the data indexes that the other output caches need to retain for each consumer and the data indexes that the producers of the other output caches will write into the other output caches next, the effective amount of data in the target input cache corresponding to the target operator to be processed is calculated.

[0209] Furthermore, the subgraph execution module to be processed includes: a second register lookup submodule and a free space calculation submodule, wherein:

[0210] The second register lookup submodule is used to look up the second register corresponding to the target operator based on the operator identifier information of the target operator to be processed.

[0211] The free space calculation submodule is used to calculate the free space in the target output buffer corresponding to the target operator to be processed based on the configuration information recorded in the second register.

[0212] Furthermore, the configuration information recorded in the second register includes the total capacity of the target output buffer, the data index that each consumer will read from the target output buffer next, and the free space that the target output buffer needs to reserve for each producer. The free space calculation submodule includes: a free space calculation unit, wherein:

[0213] The spare space calculation unit is used to calculate the spare space in the target output cache corresponding to the target operator to be processed, based on the total capacity of the target output cache, the data index that each consumer will read from the target output cache next, and the spare space that the target output cache needs to reserve for the producer.

[0214] Furthermore, a data reservation instruction is inserted between the inspection instruction and the target operator to be processed. The subgraph execution module to be processed further includes: a data reservation instruction execution submodule, wherein:

[0215] The data reservation instruction execution submodule is used to execute the data reservation instruction, and update the data index that needs to be reserved for each consumer in the output cache recorded in the corresponding register, as well as the empty space that needs to be reserved for the producer in the output cache, according to the data reservation instruction and the operator identifier of the target operator to be processed.

[0216] Furthermore, an operator update instruction is inserted after the operator to be processed, and the subgraph execution module further includes: an operator update instruction execution submodule, wherein:

[0217] The operator update instruction execution submodule is used to execute the operator update instruction if the instruction corresponding to the target operator to be processed has been executed. The instruction updates the data index recorded in the corresponding register that each consumer will read from the output buffer next, and the data index that the producer will write to the output buffer next, according to the operator identifier of the target operator to be processed.

[0218] Furthermore, the instruction sequence also includes a first scoreboard update instruction, which is inserted before the second scoreboard check instruction. The instruction processing device further includes a first scoreboard update instruction execution module, wherein:

[0219] The first scoreboard update instruction execution module is used to execute the first scoreboard update instruction to update the configuration quantity of the effective data cache space in the scoreboard if it is determined that the configuration of the subgraph to be processed has been completed in the scoreboard.

[0220] Furthermore, the instruction sequence further includes a second scoreboard update instruction, which is inserted after the subgraph to be processed. The instruction processing device further includes a second scoreboard update instruction execution module, wherein:

[0221] The second scoreboard update instruction execution module is used to execute the second scoreboard update instruction if it is determined that the instruction corresponding to the plurality of pending operators has been executed, and to notify that the resources in the scoreboard corresponding to the pending subgraph have been released.

[0222] In some implementations, one embodiment of this application provides a scheduler for executing the above-described instruction processing method.

[0223] In some implementations, one embodiment of this application provides a processor, the processor including a scheduler for executing the above-described instruction processing method.

[0224] Optionally, the processor may further include a sub-computation engine and an instruction distribution unit, wherein the instruction distribution unit is used to read the subgraph to be processed and send the subgraph to be processed to the scheduler, wherein the subgraph to be processed includes multiple operators to be processed; the scheduler is used to schedule and control whether to allow the multiple operators to be processed; and the sub-computation engine is used to process the instructions corresponding to the multiple operators to be processed that are allowed by the scheduler.

[0225] In some implementations, one embodiment of this application provides a chip that may include the processor described above.

[0226] In some embodiments, one example of this application provides an electronic device that includes the processor described above.

[0227] Optionally, the aforementioned electronic device may also include a memory, a sensor, a display, etc., without limitation.

[0228] In summary, the instruction processing method, scheduler, processor, chip, and electronic device provided in this application first execute a first scoreboard check instruction to check whether there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed. If it is found that there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed, a subgraph configuration instruction is executed to configure the subgraph to be processed in the scoreboard. The subgraph configuration instruction is determined based on the data dependencies between the multiple operators to be processed included in the subgraph to be processed. Then, a second scoreboard check instruction is executed to check whether the subgraph to be processed has been configured in the scoreboard. If it is found that the subgraph to be processed has been configured in the scoreboard, the subgraph is configured according to the data dependencies between the multiple operators to be processed in the subgraph to be processed. The scheduling control uses the configuration information of the subgraph to be processed to determine whether to allow multiple operators to proceed. This configuration information includes input buffer information, output buffer information, and various counters related to the output buffer for each operator. Based on the data dependencies between operators in the subgraph, the subgraph is configured on the scoreboard, which records the input buffer information, output buffer information, and various counters related to the output buffer for each operator. This provides a concise and clear representation of the relevant data storage information, serving as a reference for determining whether to allow operators in the subgraph. This effectively resolves data dependencies between operators and improves processing efficiency.

[0229] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An instruction processing method, characterized in that, Applied to a scheduler, the method includes: Execute the first scoreboard check instruction to check if there are enough free resources on the scoreboard to support the configuration of the subgraph to be processed; If it is found that there are enough free resources in the scoreboard to support the configuration of the subgraph to be processed, then the subgraph configuration instruction is executed to configure the subgraph to be processed in the scoreboard. The subgraph configuration instruction is determined according to the data dependency relationship between the multiple operators to be processed included in the subgraph to be processed. Execute the second scoreboard check instruction to check whether the sub-graph to be processed has been configured in the scoreboard; If the subgraph to be processed is found to be configured in the scoreboard, the scheduling control is used to determine whether to allow the multiple operators to be processed based on the configuration information of the subgraph to be processed recorded in the scoreboard. The configuration information of the subgraph to be processed includes the input buffer information, output buffer information, and various counters related to the output buffer for each of the multiple operators to be processed.

2. The method according to claim 1, characterized in that, Before executing the second scoreboard check instruction to check whether the subgraph to be processed has been configured in the scoreboard, the method further includes: If it is determined that the configuration of the subgraph to be processed has been completed in the scoreboard, then the first scoreboard update instruction is executed to update the configuration quantity of the effective data cache space in the scoreboard.

3. The method according to claim 1, characterized in that, After the step of scheduling and controlling whether to allow the plurality of pending operators based on the configuration information of the pending subgraph recorded in the scoreboard, the method further includes: If it is determined that the instructions corresponding to the plurality of pending operators have been executed, then the second scoreboard update instruction is executed to notify that the resources in the scoreboard corresponding to the pending subgraph have been released.

4. The method according to claim 1, characterized in that, The execution of the first scoreboard check instruction, which checks whether there are sufficient free resources on the scoreboard to support the configuration of the subgraph to be processed, includes: Execute the first scoreboard check instruction to check whether the idle resources in the scoreboard are sufficient to support the synchronous processing of at least two subgraphs, wherein the at least two subgraphs include the subgraph to be processed, and the synchronous processing of at least two subgraphs includes the configuration of the subgraph to be processed and the execution of instructions corresponding to operators included in other subgraphs of the at least two subgraphs; If it is found that there are enough idle resources in the scoreboard to support the synchronous processing of at least two subgraphs, then it is determined that there are enough idle resources in the scoreboard to support the configuration of the subgraph to be processed, and the corresponding idle resources are allocated for the configuration of the subgraph to be processed.

5. The method according to claim 1, characterized in that, The scoreboard corresponds to multiple registers, and the subgraph configuration instructions include operator configuration instructions corresponding to each of the multiple operators to be processed. Executing the subgraph configuration instructions, and configuring the subgraphs to be processed on the scoreboard, includes: Execute the operator configuration instructions corresponding to each of the plurality of operators to be processed, and allocate one of the plurality of registers to each of the plurality of operators to be processed; In each of the allocated registers, the input buffer information and the total capacity of the output buffer corresponding to the operator to be processed are written.

6. The method according to claim 5, characterized in that, The step of scheduling and controlling whether to allow the multiple pending operators based on the configuration information of the pending subgraph recorded in the scoreboard includes: A target operator is determined from the plurality of operators to be processed, wherein the target operator is any one of the plurality of operators to be processed, an operator check instruction is inserted before the target operator, the operator check instruction carries the effective amount of data required for the execution of the target operator, and / or the free space required for the output data of the target operator; Execute the operator check instruction to query the scoreboard to determine whether the parameter information in the target cache corresponding to the target operator to be processed meets the specified parameter information, wherein the specified parameter information includes the amount of effective data in the target input cache corresponding to the target operator to be processed reaching the amount of effective data required for the execution of the target operator to be processed, and / or the free space in the target output cache corresponding to the target operator to be processed reaching the free space required for the output data of the target operator to be processed; If it is determined that the parameter information in the target cache corresponding to the target operator to be processed meets the specified parameter information, then the target operator to be processed is allowed to proceed.

7. The method according to claim 6, characterized in that, The method further includes: Based on the operator identifier information of the target operator to be processed, the input identifier information of the target input cache corresponding to the target operator to be processed is found, wherein the target input cache is other output caches corresponding to other operators, and the other operators are one or more operators to be processed other than the target operator to be processed; Based on the input identifier information of the target input buffer, determine the first register corresponding to the other operators; Based on the configuration information recorded in the first register, the amount of valid data in the target input buffer corresponding to the target operator to be processed is calculated.

8. The method according to claim 7, characterized in that, The configuration information recorded in the first register includes the data indexes that the other output buffers need to reserve for each consumer, and the data indexes that each producer of the other output buffers will write into the other output buffers next. The step of calculating the effective data volume in the target input buffer corresponding to the target operator to be processed based on the configuration information recorded in the first register includes: Based on the data indexes that the other output caches need to retain for each consumer and the data indexes that the producers of the other output caches will write into the other output caches next, the effective amount of data in the target input cache corresponding to the target operator to be processed is calculated.

9. The method according to claim 6, characterized in that, The method further includes: Based on the operator identifier information of the target operator to be processed, locate the second register corresponding to the target operator to be processed; Based on the configuration information recorded in the second register, the free space in the target output buffer corresponding to the target operator to be processed is calculated.

10. The method according to claim 9, characterized in that, The configuration information recorded in the second register includes the total capacity of the target output buffer, the data index that each consumer will read from the target output buffer next, and the free space that the target output buffer needs to reserve for each producer. The step of calculating the free space in the target output buffer corresponding to the target operator to be processed based on the configuration information recorded in the second register includes: Based on the total capacity of the target output cache, the data index that each consumer will read from the target output cache next, and the free space that the target output cache needs to reserve for the producer, calculate the free space in the target output cache corresponding to the target operator to be processed.

11. The method according to claim 6, characterized in that, A data reservation instruction is inserted between the operator check instruction and the target operator to be processed. Before releasing the target operator to be processed, the method further includes: Execute the data reservation instruction, and update the data index that needs to be reserved for each consumer and the spare space that needs to be reserved for the producer recorded in the scoreboard according to the data reservation instruction and the operator identifier of the target operator to be processed.

12. The method according to claim 6, characterized in that, An operator update instruction is inserted after the operator to be processed. After releasing the target operator to be processed, the method further includes: If the instruction corresponding to the target operator to be processed is completed, the operator update instruction is executed to update the data index to be read by each consumer and the data index to be written by each producer recorded in the scoreboard according to the operator identification information of the target operator to be processed.

13. The method according to any one of claims 1-12, characterized in that, in: The input cache information includes the number of input caches and input identifier information used to determine each input cache; The output cache information includes the number of output caches, the total capacity of each output cache, and the number of consumers corresponding to each output cache. The various counters related to the output cache include the data index that each consumer will read from the output cache next, the data index that each producer will write to the output cache next, the data index that needs to be reserved in the output cache for each consumer, and the spare space that the output cache needs to reserve for each producer.

14. A scheduler, characterized in that, The scheduler is used to execute the method as described in any one of claims 1-13.

15. A processor, characterized in that, The processor includes the scheduler as described in claim 14.

16. The processor according to claim 15, characterized in that, The processor also includes a sub-computing engine and an instruction dispatch unit; The instruction distribution unit is used to read the subgraph to be processed and send the subgraph to be processed to the scheduler, wherein the subgraph to be processed includes multiple operators to be processed; The scheduler is used to control whether to allow the multiple pending operators to proceed. The sub-computing engine is used to process the instructions corresponding to the multiple pending operators released by the scheduler.

17. A chip or electronic device, characterized in that, Includes the processor as described in claim 15 or 16.

Images