Instruction scheduling method, artificial intelligence chip, computer device and storage medium

Abstract

The application relates to an artificial intelligence technology, in particular to an instruction scheduling method, an artificial intelligence chip, a computer device and a storage medium. The method comprises the following steps: querying an instruction mapping queue in an idle state in an instruction mapping queue set; reading out at least one to-be-executed instruction set from an instruction cache when the instruction mapping queue in the idle state is queried; sequentially writing each to-be-executed instruction set into different instruction queues corresponding to the instruction mapping queue; and scheduling instructions in the to-be-executed instruction sets of different instruction queues to execution units in a parallel mode, so that each execution unit executes instructions in different to-be-executed instruction sets in parallel, and executes instructions in the same to-be-executed instruction set in sequence according to an instruction sequence. The method can improve the execution efficiency of instructions, and further improve the throughput and execution efficiency of an AI chip.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an instruction scheduling method, an artificial intelligence chip, a computer device and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence technology, artificial intelligence technology is applied to various fields, such as application to chips, to obtain AI (Artificial Intelligence) technology-based chips (i.e., AI chips), which make chips intelligent.

[0003] In AI chip design, a group of instruction scheduling architectures usually corresponds to a set of execution architectures of "matrix operation + vector operation + load operation + store operation", so in the process of completing a task, the AI chip usually executes a corresponding instruction of the task first, and then executes the next instruction after the execution of the instruction. However, using the above-mentioned way to execute the corresponding instruction makes the execution efficiency of the instruction low, and thus affects the throughput and execution efficiency of the AI chip. SUMMARY

[0004] Therefore, it is necessary to provide an instruction scheduling method, an artificial intelligence chip, a computer device and a storage medium to improve the execution efficiency of the instruction, and thus improve the throughput and execution efficiency of the AI chip.

[0005] An instruction scheduling method, the method comprising:

[0006] querying an instruction mapping queue in an idle state in an instruction mapping queue set;

[0007] when an instruction mapping queue in an idle state is queried, reading out a set of to-be-executed instructions from an instruction cache;

[0008] writing the read-out set of to-be-executed instructions into different instruction queues corresponding to the instruction mapping queue in sequence; the instruction mapping queue is used to save the identification of the set of to-be-executed instructions and the identification of the instructions in the set of to-be-executed instructions;

[0009] scheduling the instructions in the set of to-be-executed instructions of different instruction queues to execution units in a parallel manner, so that each execution unit executes the instructions in different sets of to-be-executed instructions in parallel, and executes the instructions in the same set of to-be-executed instructions in sequence according to the instruction order.

[0010] An instruction scheduling device, the device comprising:

[0011] a querying module configured to query an instruction mapping queue in an idle state in an instruction mapping queue set;

[0012] a reading module, configured to read out to-be-executed instruction sets from the instruction cache when an idle instruction mapping queue is queried;

[0013] a writing module, configured to sequentially write each of the read out to-be-executed instruction sets into different instruction queues corresponding to the instruction mapping queue; the instruction mapping queue is configured to save the identification of the to-be-executed instruction sets and the identification of instructions in the to-be-executed instruction sets;

[0014] a scheduling module, configured to schedule the instructions in the to-be-executed instruction sets of different instruction queues to execution units in a parallel manner, so that each execution unit executes the instructions in different to-be-executed instruction sets in parallel and executes the instructions in the same to-be-executed instruction set in sequence according to the instruction order.

[0015] An artificial intelligence chip, comprising a state querying unit, an instruction scheduling unit and an execution unit; the instruction scheduling unit comprises an instruction cache, an instruction gating subunit, an instruction distribution subunit and an instruction queue unit;

[0016] The state querying unit is configured to query an idle instruction mapping queue in a set of instruction mapping queues;

[0017] The instruction cache is configured to store to-be-executed instruction sets;

[0018] The instruction gating subunit is configured to read out the to-be-executed instruction sets from the instruction cache when an idle instruction mapping queue is queried;

[0019] The instruction distribution subunit is configured to sequentially write each of the read out to-be-executed instruction sets into different instruction queues corresponding to the instruction mapping queue; the instruction mapping queue is configured to save the identification of the to-be-executed instruction sets and the identification of instructions in the to-be-executed instruction sets;

[0020] The instruction queue unit is configured to schedule the instructions in the to-be-executed instruction sets of different instruction queues to execution units in a parallel manner;

[0021] The execution unit is configured to execute the instructions in different to-be-executed instruction sets in parallel and execute the instructions in the same to-be-executed instruction set in sequence according to the instruction order.

[0022] A computer device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0023] querying an idle instruction mapping queue in a set of instruction mapping queues;

[0024] When an idle instruction mapping queue is queried, a set of to-be-executed instructions is read from the instruction cache;

[0025] Each of the read sets of to-be-executed instructions is sequentially written into a different instruction queue corresponding to the instruction mapping queue; the instruction mapping queue is used to save the identification of the set of to-be-executed instructions and the identification of instructions in the set of to-be-executed instructions;

[0026] The instructions in the sets of to-be-executed instructions of different instruction queues are scheduled to execution units in a parallel manner, so that each execution unit executes instructions in different sets of to-be-executed instructions in parallel and executes instructions in the same set of to-be-executed instructions in sequence according to the order of the instructions.

[0027] A computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the following steps:

[0028] An idle instruction mapping queue is queried in a set of instruction mapping queues;

[0029] When an idle instruction mapping queue is queried, a set of to-be-executed instructions is read from the instruction cache;

[0030] Each of the read sets of to-be-executed instructions is sequentially written into a different instruction queue corresponding to the instruction mapping queue; the instruction mapping queue is used to save the identification of the set of to-be-executed instructions and the identification of instructions in the set of to-be-executed instructions;

[0031] The instructions in the sets of to-be-executed instructions of different instruction queues are scheduled to execution units in a parallel manner, so that each execution unit executes instructions in different sets of to-be-executed instructions in parallel and executes instructions in the same set of to-be-executed instructions in sequence according to the order of the instructions.

[0032] The instruction scheduling method, device, artificial intelligence chip, computer device and storage medium described above, by querying whether there is an idle instruction mapping queue in a set of instruction mapping queues, when an idle instruction mapping queue is queried, it indicates that there is an instruction queue corresponding to the instruction mapping queue and in an idle state, so at least one set of to-be-executed instructions can be read from the instruction cache, then the identification of the set of to-be-executed instructions and the identification of instructions in the set of to-be-executed instructions are saved in the instruction mapping queue, and each set of to-be-executed instructions is sequentially written into a different instruction queue, so that there are instruction queues that can be scheduled in parallel, instructions between different instruction queues can be scheduled in parallel and executed by execution units, and instructions in the same instruction queue can be sequentially scheduled and executed by execution units, so that the execution efficiency of instructions can be improved, and the probability of using execution units can be improved, thereby effectively improving the throughput and execution efficiency of the AI chip. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 An application environment diagram of the instruction scheduling method in one embodiment;

[0034] Figure 2 A flowchart of the instruction scheduling method in one embodiment;

[0035] Figure 3 A structural diagram of the artificial intelligence chip in one embodiment;

[0036] Figure 4 A diagram of video decoding and playing by the artificial intelligence chip in one embodiment;

[0037] Figure 5 A flowchart of the instruction reading step from the instruction cache in one embodiment;

[0038] Figure 6 A flowchart of the instruction reading step from the memory in one embodiment;

[0039] Figure 7 A flowchart of the instruction mapping queue and instruction queue cleaning step when the instruction execution is completed in one embodiment;

[0040] Figure 8 An application environment diagram of the instruction scheduling method in another embodiment;

[0041] Figure 9 A structural block diagram of the instruction scheduling device in one embodiment;

[0042] Figure 10 A structural block diagram of the instruction scheduling device in another embodiment;

[0043] Figure 11 An internal structural diagram of the computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0045] Artificial Intelligence (AI) is the use of digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0046] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning, etc.

[0047] With the research and progress of artificial intelligence technology, artificial intelligence technology has been researched and applied in many fields, such as common smart home, smart wearable device, virtual assistant, smart speaker, smart marketing, unmanned vehicle, autonomous vehicle, unmanned aerial vehicle, robot, smart medical treatment, smart customer service, etc. It is believed that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0048] The scheme provided by the embodiments of the present application relates to artificial intelligence chips and terminals integrating artificial intelligence chips, which are specifically explained as follows:

[0049] The instruction scheduling method provided by the present application can be applied to, for example Figure 1The application environment shown. In this application environment, including terminal 102, terminal 104 and server 106. Terminal 104 or server 106 can initiate one or more target tasks to terminal 102, or initiate one or more target tasks inside terminal 102, when terminal 102 gets the target task, it can compile the computer program corresponding to the target task to get a large number of instructions, which can be stored in on-chip or off-chip memory, terminal 102 reads the instructions to be executed from the memory, and then queries the idle instruction mapping queue in the instruction mapping queue set; When an idle instruction mapping queue is queried, read the to-be-executed instruction set from the instruction cache; Write the read each to-be-executed instruction set to different instruction queue corresponding to the instruction mapping queue in turn; The instruction mapping queue is used to save the identification of the to-be-executed instruction set and the identification of the instruction in the to-be-executed instruction set; In a parallel manner, the instructions in the to-be-executed instruction set of different instruction queues are scheduled to the execution unit, so that each execution unit executes the instructions in different to-be-executed instruction sets in parallel, and executes the instructions in the same to-be-executed instruction set in turn according to the instruction order.

[0050] Wherein, the terminal 102 and the terminal 104 can be various electronic devices integrated with artificial intelligence chips, such as smart phones, tablet computers, notebook computers, desktop computers and smart watches, as well as printers and smart home appliances, etc., and the smart home appliances can specifically include smart speakers or televisions, etc., but are not limited thereto. The above-mentioned artificial intelligence chip can refer to an AI processor, or other AI chips, such as AI circuit.

[0051] The server 106 can be a stand-alone physical server, or a server cluster composed of multiple physical servers, and can be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms, etc. Basic cloud computing services.

[0052] The terminal 102, the terminal 104 and the server 106 can be connected through Bluetooth, USB (Universal Serial Bus, Universal Serial Bus) or network communication connection mode, which is not limited in the present application.

[0053] In one embodiment, as Figure 2 shown, an instruction scheduling method is provided, which is applied to the terminal in Figure 1 for example, including the following steps:

[0054] S202, query the idle instruction mapping queue in the instruction mapping queue set.

[0055] The instruction mapping queue can be used to store the identifier of the instruction, the execution state, and the identifier of the instruction set to be executed where the instruction is located. The instruction mapping queue and the instruction queue have a mapping relationship. In terms of function, the instruction mapping queue can be used as a mapping of the instruction queue. Therefore, the instruction mapping queue can also be referred to as the instruction queue mapping. In subsequent embodiments, the two can be referred to each other. If there are n instruction queues, there are n instruction mapping queues. When an instruction mapping queue is queried, the corresponding instruction queue can be found according to the instruction mapping queue. In addition, when the identifier of the instruction in the instruction mapping queue is queried, the instruction corresponding to the identifier can be found in the instruction queue. Moreover, when the identifier of the instruction set to be executed in the instruction mapping queue is queried, the instruction set corresponding to the identifier can be found in the instruction queue. In the instruction mapping queue, the identifier, the execution state, and the identifier of the instruction set to be executed can be processed in a first input first output (FIFO) manner or an instruction pointer manner. When the instruction is executed, the identifier and the execution state corresponding to the instruction can be deleted in the instruction mapping queue, or the execution state corresponding to the instruction can be marked as an executed state in the instruction mapping queue. Correspondingly, the instruction mapping queue set is a set composed of multiple instruction mapping queues.

[0056] The execution state includes an unexecuted state and an executed state. The unexecuted state indicates that the instruction is in an unexecuted state, and the executed state indicates that the instruction is in an executed state.

[0057] The instruction mapping queue in the idle state can mean that the instruction mapping queue does not store information, that is, the instruction mapping queue is empty.

[0058] In addition, the instruction queue can be a queue used to store instructions. In the instruction queue, the instructions can be read in a first input first output manner or an instruction pointer manner.

[0059] In one embodiment, the terminal integrates an artificial intelligence chip, and the artificial intelligence chip includes an instruction scheduling unit, a state query unit, and an execution unit, as shown in FIG. 1. Figure 3 The instruction scheduling unit can realize reading and scheduling (that is, distribution) of the instructions. The state query unit can update the completion of the execution of the instructions in real time. The execution unit can execute the instructions to complete the operations corresponding to the instructions and return the completion of the instructions.

[0060] Specifically, the terminal can query the instruction mapping queue in idle state in the instruction mapping queue set through the state query unit. The state query unit includes a queue state query subunit and at least one instruction queue mapping subunit. In addition, the instruction scheduling unit includes an instruction cache, an instruction gating subunit, an instruction distribution subunit, and an instruction queue unit, as shown in Figure 3 Therefore, when the instruction gating subunit of the terminal needs to read instructions from the instruction cache, it needs to query the instruction mapping queue in idle state in the instruction mapping queue set through the queue state query subunit. If an idle instruction mapping queue is found, S204 is performed.

[0061] For example, when there is a new set of instructions to be executed in the instruction cache, the set of instructions to be executed needs to be distributed and stored in the instruction queue. At this time, the instruction gating subunit sequentially finds instruction mapping queues 1-n according to the size of the number through the state query unit. When the first empty instruction mapping queue is found, assuming that the number of the empty instruction mapping queue is i, it is indicated that the i-th instruction queue is empty, and S204 is performed to store the identifier of the set of instructions to be executed and the identifier of the instruction in the i-th instruction mapping queue. In addition, the execution state of the instruction in the set of instructions to be executed is also stored. The set of instructions to be executed is sequentially written into the corresponding instruction queue.

[0062] S204, when an idle instruction mapping queue is found, read at least one set of instructions to be executed from the instruction cache.

[0063] The instruction cache can be a cache for storing instructions in an artificial intelligence chip. The instruction cache belongs to the first-level cache of the artificial intelligence chip. In addition, the data cache also belongs to the first-level cache of the artificial intelligence chip and is used to store data.

[0064] In an embodiment, for each set of instructions to be executed, the terminal can sequentially read the instructions in each set of instructions to be executed from the instruction cache to obtain at least one set of instructions to be executed. The set of instructions to be executed can be a set of instructions to be executed obtained by compiling a computer program corresponding to a target task when the target task needs to be completed. The instructions can be coarse-grained instructions or fine-grained instructions. Fine-grained instructions can refer to instructions for completing low-level or small-grained tasks, such as instructions for performing addition, subtraction, multiplication, and division tasks, or instructions for performing logical operation tasks, etc. Fine-grained instructions can be completed in 1 to 2 clock cycles during execution, and a pipeline execution of value taking, decoding, execution, and write-back is used. Coarse-grained instructions can refer to instructions for completing high-level or large-grained tasks, such as instructions for performing video encoding and decoding tasks, or instructions for performing image encoding and decoding tasks, etc. These high-level or large-grained tasks can be split into multiple low-level or small-grained tasks, or these high-level or large-grained tasks can be split into multiple sub-high-level or sub-large-grained tasks, and then split into multiple low-level or small-grained tasks. For example, a video encoding and decoding task can be split into a large number of addition, subtraction, multiplication, and division tasks and logical operation tasks. For another example, a task of making Yangzhou fried rice by a smart home appliance device is a large-grained task. The task of making Yangzhou fried rice can be split into tasks of putting oil, frying eggs, frying ham and shrimp, and frying rice, which are fine-grained tasks. During execution of coarse-grained instructions, each instruction usually executes more than tens of clock cycles, so even if the pipeline of value taking, decoding, execution, and write-back is not used, the performance is not greatly affected. From the above description, it can be seen that, compared with the execution time of instructions, the time consumed by the acquisition and decoding operations of coarse-grained instructions can be ignored, and the time consumed by the acquisition and decoding operations of fine-grained instructions cannot be ignored. For the convenience of description, unless otherwise specified, the instructions and instructions in specific application scenarios (such as video decoding instructions) appearing in subsequent embodiments all refer to coarse-grained instructions.

[0065] When one or more target tasks need to be completed, the terminal can compile a computer program of the target task to obtain corresponding instructions, and then store the obtained instructions in the instruction cache according to different target tasks or sub-tasks included in the target tasks in the corresponding set of instructions to be executed. The set of instructions to be executed can be a set created by the terminal in the instruction cache for storing instructions. When an idle instruction mapping queue is queried, instructions in the set of instructions to be executed are read from the instruction cache based on a first-in first-out manner or an instruction pointer manner to obtain at least one set of instructions to be executed. The instruction pointer is also called a program counter (PC) pointer.

[0066] The first-in first-out mode can refer to that, for instructions in the instruction cache, the instructions can be read from the instruction cache according to the writing order of the instructions.

[0067] The instruction pointer mode can refer to that, the corresponding instruction is read by using the instruction pointer. The instruction pointer is used to point to the address of an instruction being executed or an instruction to be executed, and is automatically incremented by one after the execution of the instruction to point to the next instruction.

[0068] For example, as shown in FIG. 6, when the instruction gating subunit of the terminal queries the instruction mapping queue in the idle state through the queue state query unit, the terminal reads the instructions in the to-be-executed instruction set from the instruction cache in sequence through the instruction gating subunit, and obtains at least one to-be-executed instruction set. Figure 3

[0069] In one embodiment, before reading the at least one to-be-executed instruction set, the terminal further queries the execution status of a target instruction set having a dependency relationship with the at least one to-be-executed instruction set, and specifically: the terminal queries the target instruction set that has been written into another instruction queue and has a dependency relationship with the at least one to-be-executed instruction set; obtains the execution status of each instruction in the target instruction set; and if the execution status is the executed state, the terminal reads the instructions in the at least one to-be-executed instruction set from the instruction cache based on the first-in first-out mode or the instruction pointer mode.

[0070] S206, sequentially write each to-be-executed instruction set into different instruction queues corresponding to the instruction mapping queue; the instruction mapping queue is used to save the identifier of the to-be-executed instruction set and the identifier of the instruction in the to-be-executed instruction set.

[0071] The instruction mapping queue corresponds to the instruction queue, when there is at least one instruction mapping queue in the state query unit, there is at least one instruction queue in the instruction scheduling unit, and the number of instruction mapping queues is consistent with the number of instruction queues. Generally, the instruction mapping queue and the instruction queue with the same number correspond to each other, for example, the ith instruction mapping queue corresponds to the ith instruction queue, when the ith instruction mapping queue is in the idle state, the corresponding ith instruction queue is also in the idle state. It should be noted that the idle state can mean that the instruction queue and the instruction mapping queue are empty queues.

[0072] In one embodiment, S206 can specifically include that, when the terminal reads the instructions in different to-be-executed instruction sets from the instruction cache, the terminal can write the instructions read each time into different instruction queues corresponding to the instruction mapping queue.

[0073] ​Before writing the instructions corresponding to the instruction mapping queue, the terminal can first determine the number of the instruction mapping queue, find the instruction queue matched with the number according to the number of the instruction mapping queue, and then write the sets of to-be-executed instructions into the different instruction queues found.

[0074] For example, as shown in FIG. 2, if it is found that the instruction mapping queue 1 and the instruction mapping queue 2 are in the idle state, it is determined that the instruction queue 1 and the instruction queue 2 are also in the idle state. At this time, the terminal sequentially reads the instructions from the two sets of to-be-executed instructions in the instruction cache to obtain the two sets of to-be-executed instructions read. Then, the terminal writes the instructions in the two sets of to-be-executed instructions into the instruction queue 1 and the instruction queue 2, respectively. Figure 3

[0075] For the way of writing the sets of to-be-executed instructions into the instruction queue, there can be two ways, i.e., reading and writing at the same time and writing after reading. Specifically, the terminal can write the set of to-be-executed instructions obtained when reading the instructions in the set of to-be-executed instructions from the instruction cache into the instruction queue. In addition, the terminal can write the instructions read in the process of reading the set of to-be-executed instructions from the instruction cache into the instruction queue.

[0076] S208, in a parallel manner, dispatch the instructions in the sets of to-be-executed instructions of different instruction queues to the execution units, so that each execution unit executes the instructions in different sets of to-be-executed instructions in parallel and executes the instructions in the same set of to-be-executed instructions in sequence according to the order of the instructions.

[0077] It can be understood that the execution of the dispatched instructions by the execution unit can refer to the completion of the corresponding target task by the execution unit according to the dispatched instructions. For example, assuming that the instructions are video decoding instructions, the execution unit can complete video decoding according to the dispatched multiple video decoding instructions, so as to play after decoding the video.

[0078] The number of execution units can be one or more. When there are multiple execution units, the execution units can be execution units of multiple functions, such as execution units for completing video decoding, execution units for completing audio decoding, or execution units for completing image rendering, etc. In addition, there can be multiple execution units of each function.

[0079] ​In the execution process, the target task to be completed can be divided into a plurality of sub-tasks according to the granularity size, each to-be-executed instruction set stores instructions of a corresponding sub-task, the instruction mapping queue stores the identifier of the to-be-executed instruction set corresponding to the sub-task and the identifier of each instruction in the to-be-executed instruction set, and in addition, stores the execution state corresponding to each instruction. Therefore, in scheduling and execution, the instructions in different to-be-executed instruction sets that do not have a dependency relationship can be scheduled in parallel, so that different execution units execute the scheduled instructions in parallel, and the instructions in a single to-be-executed instruction set are executed in sequence. In addition, for different to-be-executed instruction sets that have a dependency relationship, the instructions executed first can be scheduled for execution first, and after execution is completed, the instructions executed later can be scheduled for execution.

[0080] Among them, the dependency relationship refers to that one to-be-executed instruction set needs to wait for one or more preceding target instruction sets to be completed before execution. For example, assuming that there are the following 5 to-be-executed instruction sets:

[0081] To-be-executed instruction set A: rlt1 = a + b * c;

[0082] To-be-executed instruction set B: rlt2 = c + d * e;

[0083] To-be-executed instruction set C: rlt3 = rlt1 + f + g;

[0084] To-be-executed instruction set D: rlt4 = rlt1 + rlt2 * rlt3;

[0085] To-be-executed instruction set E: rlt5 = h + i * c;

[0086] As can be seen from the above, the execution of the to-be-executed instruction set C depends on the to-be-executed instruction set A, the execution of the to-be-executed instruction set D depends on the to-be-executed instruction sets A, B, and C, and the to-be-executed instruction sets A, B, and E do not depend on each other. Therefore, the compiler optimizes the output arrangement order to be: {A B E, C, D}, that is, the to-be-executed instruction sets A, B, and E can be executed simultaneously first, then the to-be-executed instruction set C is executed, and finally the to-be-executed instruction set D is executed.

[0087] In an embodiment, the number of instruction queues or the number of instruction sets can be greater than the number of execution units. When it is greater than the number of execution units, for the instruction set written to the instruction queue first, the instructions thereof can be scheduled to the execution unit first, thereby preempting the execution unit for execution. In addition, the number of instruction queues or the number of instruction sets can be equal to the number of execution units, and when it is equal to the number of execution units, the instructions in the instruction set of each instruction queue can be scheduled to the execution unit simultaneously for execution.

[0088] Next, the above instruction scheduling method is described in combination with the application scenario of a video decoding task, which can be divided into multiple video decoding sub-tasks, as shown in Figure 4 As shown in FIG. 8, the terminal compiles a computer program of a video decoding task, and the obtained multiple video decoding instructions are saved in a memory; an instruction scheduling unit sequentially reads the saved video decoding instructions from the memory and writes the read video decoding instructions into an instruction cache. The instruction scheduling unit queries whether there is an idle instruction queue mapping through a state querying unit, and if multiple idle instruction queue mappings are found, the instruction scheduling unit reads the video decoding instructions from the instruction cache to obtain multiple to-be-executed instruction sets containing the video decoding instructions. Then, the video decoding instructions in the multiple to-be-executed instruction sets are respectively stored in instruction queues, and the identification of the multiple to-be-executed instruction sets, the identification of the instructions in the to-be-executed instruction sets, and the execution state of the instructions are stored in an instruction mapping queue. The terminal schedules the instructions in the to-be-executed instruction sets in the multiple instruction queues to multiple execution units in parallel, so that the execution units decode the video according to the video decoding instructions, and then transmit the decoded video to a client for playing.

[0089] In the above embodiment, whether there is an idle instruction mapping queue in the instruction mapping queue set is queried, and when an idle instruction mapping queue is found, it indicates that there is an idle instruction queue corresponding to the instruction mapping queue. Therefore, at least one to-be-executed instruction set can be read from the instruction cache, and then the identification of the to-be-executed instruction set and the identification of the instructions in the to-be-executed instruction set are saved in the instruction mapping queue, and each to-be-executed instruction set is sequentially written into a different instruction queue. Therefore, the instruction queue that can be executed in parallel can schedule instructions, the instructions between different instruction queues can be executed in parallel and executed by the execution unit, the instructions in the same instruction queue can be sequentially scheduled and executed by the execution unit, so that the execution efficiency of the instructions can be improved, the probability of using the execution unit can be improved, and the throughput and execution efficiency of the AI chip are effectively improved.

[0090] In one embodiment, as shown in Figure 5 The method can further include:

[0091] S502, sequentially read the instructions obtained by compiling the computer program from the memory.

[0092] The instructions include coarse-grained instructions. The computer program can refer to a program developed by a programming language to achieve various target tasks or functions. For example, for a video decoding task, the computer program can be a program related to video decoding.

[0093] In one embodiment, upon receiving the target task, the terminal searches for a program file corresponding to the target task, compiles the computer program in the program file, obtains corresponding instructions, and then executes S502. The memory may be an on-chip memory or an off-chip memory.

[0094] S504: Write the read instruction into the corresponding instruction set to be executed in the instruction cache.

[0095] Specifically, when performing a write operation, the terminal can classify and write the obtained instructions according to the granularity of the target task, that is, write the instructions belonging to the same subtask into the same set of instructions to be executed in the instruction cache, and write the instructions belonging to different subtasks into different sets of instructions to be executed in the instruction cache. Alternatively, it can be understood that the instructions belonging to the same subtask are written into the instruction cache with the same set of instructions to be executed.

[0096] S506 , reading instructions in the instruction set to be executed from the instruction cache based on a first-in-first-out method or an instruction pointer method.

[0097] The above-mentioned first-in-first-out method may mean that, for instructions written into the instruction cache, the instructions may be read out from the instruction cache in the order in which the instructions were written.

[0098] The instruction pointer method mentioned above can be used to read the corresponding instruction through the instruction pointer. The instruction pointer is used to point to the address of the instruction being executed or the address of the instruction to be executed, and automatically increases by one after execution to point to the next instruction.

[0099] For example, Figure 3 As shown, when the instruction gating subunit of the terminal queries the instruction mapping queue in the idle state through the queue state query unit, the terminal sequentially reads the instructions in the instruction set to be executed from the instruction cache through the instruction gating subunit.

[0100] In one embodiment, before S506, the method may further include: the terminal queries a target instruction set that has been written into other instruction queues and has a dependency relationship with the instruction set to be executed; obtains the execution status of each instruction in the target instruction set; if the execution status is an executed state, executes S506.

[0101] Since there may be dependencies between the instruction sets corresponding to different tasks or subtasks, isolation instructions (Fence instructions) are added between different instruction sets with dependencies, so as to determine whether the current instruction set to be executed depends on the scheduled target instruction set based on the isolation instructions, so that when all the instructions in the target instruction set are executed, the instruction set to be executed is read from the instruction cache and written into the corresponding instruction queue.

[0102] For example, the following describes the to-be-executed instruction sets A-E in the above embodiment:

[0103] To-be-executed instruction set A: rlt1 = a + b * c;

[0104] To-be-executed instruction set B: rlt2 = c + d * e;

[0105] To-be-executed instruction set C: rlt3 = rlt1 + f + g;

[0106] To-be-executed instruction set D: rlt4 = rlt1 + rlt2 * rlt3;

[0107] To-be-executed instruction set E: rlt5 = h + i * c;

[0108] Since there is no dependency relationship between the to-be-executed instruction sets A, B, and E, no isolation instruction needs to be added between the to-be-executed instruction sets A, B, and E; and since there is a dependency relationship between the to-be-executed instruction set A and the to-be-executed instruction set C, an isolation instruction can be added between the to-be-executed instruction set C and the to-be-executed instruction sets A, B, and E, such as {A B E S0 C}, where S0 is an isolation instruction; and since there is a dependency relationship between the to-be-executed instruction set D and the to-be-executed instruction sets A, B, and C, an isolation instruction can be added between the to-be-executed instruction set D and the to-be-executed instruction set C, such as {A B E S0 C S1 D}, where S1 is also an isolation instruction.

[0109] In one embodiment, when an isolation instruction is added between to-be-executed instruction sets corresponding to different tasks or sub-tasks and having a dependency relationship, the step of querying a target instruction set having a dependency relationship with the to-be-executed instruction set and written into the other instruction queue can specifically include: the terminal searching for an isolation instruction corresponding to the to-be-executed instruction set; reading a target instruction set identifier from the isolation instruction; and querying the target instruction set identifier having a dependency relationship with the to-be-executed instruction set in the other instruction queue according to the target instruction set identifier.

[0110] For example, the following continues to describe the to-be-executed instruction sets A-E in the above embodiment, since an isolation instruction is added between the to-be-executed instruction sets having a dependency relationship, and assuming that the to-be-executed instruction sets with the added isolation instruction are {A B E S0 C S1 D}, when reading the to-be-executed instruction sets from the instruction cache, it is first determined according to the isolation instruction whether the to-be-executed instruction set (i.e., the above target instruction set) that needs to be executed first has been executed, and if so, the instruction in the to-be-executed instruction set is read from the instruction cache based on a first-in first-out manner or an instruction pointer manner.

[0111] In one embodiment, the step of obtaining the execution status of each instruction in the target instruction set can specifically include: searching, by the terminal, a target instruction mapping queue in which the target instruction set identifier is stored in the instruction mapping queue set; and obtaining the execution status of each instruction in the target instruction mapping queue.

[0112] In the above embodiment, when reading the to-be-executed instruction set from the instruction cache, it is necessary to determine whether the target instruction set having a dependency relationship with the to-be-executed instruction set is executed. Only when the target instruction set is executed, the instructions in the to-be-executed instruction set are read from the instruction cache based on the first-in first-out mode or the instruction pointer mode. On the one hand, it can be ensured that the to-be-executed instruction set can be executed smoothly. On the other hand, it can avoid the situation that other to-be-executed instruction sets cannot use the instruction set and the execution unit due to the occupation of the instruction set and the execution unit by the to-be-executed instruction set, thereby effectively improving the instruction execution efficiency.

[0113] In one embodiment, as shown in Figure 6 the method can further include:

[0114] S602, compiling a computer program to obtain corresponding instructions.

[0115] S604, saving the obtained instructions to a memory.

[0116] S606, sequentially reading the saved instructions from the memory after the processor is powered on or after the preliminary data required for executing the obtained instructions is obtained.

[0117] The preliminary data can refer to data required for executing the obtained instructions. For example, for decoding a video, the preliminary data can be video data to be decoded. The processor can include an artificial intelligence processor.

[0118] In one embodiment, the method further includes: if the instructions saved in the memory depend on preliminary data when executed, the terminal obtains and loads the preliminary data from the memory or a target cache; and the preliminary data includes data generated by the executed instructions.

[0119] For the data generated by the executed instructions, the generated data can be stored from the data cache close to the computing subunit to the data cache far away, such as from the L1 data cache to the L3 data cache, or the generated data can be stored from the data cache close to the memory. The execution unit includes an instruction forwarding subunit and a function subunit, and can further include a computing subunit. The computing subunit can be located between the instruction forwarding subunit and the function unit (FuncEngine) for executing instructions.

[0120] Before executing the instructions in the to-be-executed instruction set, if the generated data needs to be used when executing the instructions in the to-be-executed instruction set, data reading can be performed through a load operation, that is, writing from a data cache or a memory far away from the computing unit to a data cache close to the computing unit, such as reading from an L3 data cache to an L1 data cache, or writing from a DDR (Double Data Rate) memory to an L3 data cache.

[0121] In the above embodiment, when executing the instructions in the to-be-executed instruction set, the data necessary for executing the to-be-executed instruction is acquired first, and then the instructions in the to-be-executed instruction set are executed, which can ensure that the instructions in the to-be-executed instruction set can be executed smoothly, and can avoid other to-be-executed instruction sets from being unable to use the instruction set and the execution unit due to the to-be-executed instruction set occupying the instruction queue and the execution unit, thereby effectively improving the instruction execution efficiency.

[0122] In one embodiment, as shown in Figure 7 the method can further include:

[0123] S702, detecting an execution state of an instruction in an instruction queue.

[0124] The execution state can include an unexecuted state and an executed state. The unexecuted state indicates that the instruction in the instruction queue is not executed, and the executed state indicates that the instruction in the instruction queue is executed.

[0125] S704a, when it is determined according to the execution state that a target instruction in the instruction queue is executed, clearing the target instruction in the instruction queue.

[0126] S706a, deleting an identifier of the target instruction in an instruction mapping queue associated with the instruction queue.

[0127] The instruction and the target instruction include coarse-grained instructions. The target instruction can be an executed instruction in the instruction queue. Through the steps of S706a and S706b, the target instruction in the instruction queue can be cleared and the identifier of the target instruction in the instruction mapping queue can be deleted each time a target instruction is executed.

[0128] S704b, when it is determined according to the execution state that a target instruction in the instruction queue is executed, marking the target instruction as an executed state in an instruction mapping queue associated with the instruction queue.

[0129] S706b, when all instructions in the instruction queue are executed, cleaning the instruction mapping queue associated with the instruction queue.

[0130] S708, clearing all instructions in the instruction queue.

[0131] Among them, through the above steps S706a and S706b, each time a target instruction is executed, the target instruction can be marked as executed in the instruction mapping queue (that is, updated from the unexecuted state to the executed state). When all instructions in the to-be-executed instruction set in the instruction queue are executed, the to-be-executed instruction set in the instruction queue can be cleared, and the identifier of the to-be-executed instruction set in the instruction mapping queue, the identifier of the instructions in the to-be-executed instruction set and the execution status can be deleted.

[0132] For all target instruction queues in the occupied state, in one embodiment, when the target instruction corresponding to the target instruction mapping queue in the occupied state in the instruction mapping queue set is completed, the terminal sets the execution status of the target instruction in the target instruction mapping queue to the executed state; when the execution status of each instruction in the target instruction mapping queue is the executed state, the target instruction mapping queue is cleared.

[0133] In the above embodiment, by deleting the identifier of the executed instruction from the instruction mapping queue and deleting the executed instruction from the execution queue, it is possible to avoid the executed instruction occupying cache space, which is conducive to improving the effective utilization of the cache space.

[0134] In order to make it easier to understand the above instruction scheduling method, it is explained in combination with the application scenario of the smart home system: for example, Figure 8 As shown, terminal 804 can be a smart phone, and terminal 802 can be a smart TV. The user's smart phone sends a video playback task to the smart TV through a base station signal or a wireless local area network (such as WiFi). The AI ​​chip built into the smart TV compiles the video decoding program to obtain instructions, and then saves the obtained instructions in the instruction cache to obtain an instruction set to be executed containing instructions; wherein, the instruction set to be executed can be an instruction set corresponding to multiple decoding subtasks, that is, multiple instruction sets for decoding videos. The AI ​​chip queries the instruction mapping queue set for an idle instruction mapping queue; when an idle instruction mapping queue is queried, the instruction set to be executed is read from the instruction cache; each instruction set to be executed is written into different instruction queues in sequence; the instructions in the instruction sets to be executed of different instruction queues are scheduled to the execution units in parallel, so that each execution unit executes the video decoding instructions in different instruction sets to be executed in parallel, and executes the video decoding instructions in the same instruction set to be executed in sequence according to the instruction order, thereby completing the decoding of the video, and then playing the video through the client on the smart TV.

[0135] As another example, coarse-grained instructions (hereinafter referred to as instructions) are described, which are applicable to: FPGA / ASIC processors based on coarse-grained instructions, such as AI processors or other processors.

[0136] Application scenario: in an AI processor, the instruction cache sequentially reads the instructions to be executed from the off-chip or on-chip memory and sequentially sends them to the instruction scheduling unit, which sends the instructions to the execution unit to implement the corresponding operations of the instructions. When the instruction execution is completed, the execution unit sends a signal to the instruction scheduling unit that the coarse-grained instruction is completed. In addition, the execution unit also includes one or more load or store functional sub-units that can read or write access the off-chip or on-chip memory. In the above process, the instructions in the instruction cache have the following characteristics:

[0137] 1) The instructions are coarse-grained instructions, for example, one instruction can execute for more than tens of clock cycles;

[0138] 2) One instruction completes at least one operation, and one task can be composed of one or more instructions;

[0139] 3) If a task is composed of multiple instructions, these instructions may have dependencies and data dependencies;

[0140] 4) Multiple tasks may or may not have data dependencies. For the case where there is no data dependency, they can be executed in parallel. If in a functional unit, instructions from multiple instruction fragments that do not have data dependencies are received, these instructions can be executed out of order.

[0141] Next, the overall flow of the instruction scheduling method is introduced: the compilation tool compiles the computer program of all tasks to be executed to obtain the sequence of instructions to be executed; before the AI processor is running, the instructions in the sequence of instructions to be executed are stored in the on-chip or off-chip memory of the AI processor as the instructions to be executed by the AI processor. After the AI processor is powered on or the instruction execution condition is met (such as obtaining the necessary data to be processed), the instructions are read from the on-chip or off-chip memory to the instruction cache in one or multiple times, and then executed.

[0142] Next, the architecture involved in the instruction scheduling method and the functions and operations of each unit in the architecture are introduced as follows:

[0143] (1) Architecture description: The architecture is generally divided into three parts: instruction scheduling unit, state query unit, and execution unit. The instruction scheduling unit implements the reading and distribution of instructions, the state query unit is used to update the completion of instruction execution in real time, and the execution unit completes the operations corresponding to each instruction and returns the instruction completion status.

[0144] (2) Instruction scheduling unit, sequentially read instructions in on-chip or off-chip memory, and then write into instruction cache. Among them, instruction cache can use FIFO or cache (for example, using pc pointer) and other ways, and instruction gating sub-unit sequentially read out instructions. Instructions can be read in the form of instruction fragments (corresponding to the above instruction set), each instruction fragment contains one or more instructions, and instructions belonging to the same instruction fragment are read out continuously. After reading an instruction fragment, and before sending to instruction distribution sub-unit, it needs to query whether there is an empty instruction queue mapping through state query unit. If so, the instruction gating sub-unit reads the instruction fragment from the instruction cache, and sends it to the corresponding instruction queue through the instruction distribution sub-unit, while saving the instruction related record in the instruction queue mapping sub-unit corresponding to the state query unit. The instruction related record contains instruction fragment id and sub-id record of each instruction. Instruction scheduling unit contains n instruction queues, each instruction queue can store a to-be-executed instruction fragment, and the internal instruction is executed in order. There is no data dependency between the n instruction fragments corresponding to the n instruction queues, and they can be executed in parallel.

[0145] The instruction gating sub-unit in the instruction scheduling unit is used to control whether the current instruction fragment is issued. Its judgment rule is: if the instruction fragment to be issued needs to wait for a number of issued instruction fragments to be completed before it can be issued, the compiler inserts a synchronization instruction (Fence instruction) before the instruction fragment to be issued. The synchronization instruction is located between two instruction fragments, and the parameter in the synchronization instruction identifies one or more instruction fragment IDs, indicating that the next instruction fragment in the instruction cache needs to wait for all these instruction fragments to be executed before it can continue to be dispatched. At this time, the completion status of the specified instruction fragment in each instruction queue needs to be detected through the query operation of the state query unit.

[0146] (3) The queue state query sub-unit in the state query unit provides state query of each instruction queue. It is connected with n instruction queue mapping sub-units below, each instruction queue mapping corresponds to an instruction queue in the instruction scheduling unit, and stores the completion of the instructions dispatched to the instruction queue. In the execution unit, whenever an instruction is completed, a completion signal is sent to the corresponding instruction queue mapping sub-unit in the state recording sub-unit. When all instructions in a non-empty instruction queue mapping sub-unit are completed, the completion signal of the instruction queue mapping sub-unit is set to one, for the instruction scheduling unit to query.

[0147] (4) The execution unit receives instructions dispatched from multiple instruction queues, executes them internally, and then returns a completion signal to the state query unit. Among them, the execution unit can be 1 execution unit, i execution units with different functions, or multiple execution engines in each of i execution units with different functions, where i is a positive integer greater than 1.

[0148] It should be understood that although Figure 2 、 5 The steps in the flowchart of -7 are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 、 5 -At least part of the steps in 7 may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0149] In one embodiment, Figure 9 As shown, an instruction scheduling device is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes: a query module 902, a reading module 904, a writing module 906 and a scheduling module 908, wherein:

[0150] A query module 902 is configured to query an idle instruction mapping queue in the instruction mapping queue set;

[0151] The reading module 904 is configured to read at least one instruction set to be executed from the instruction cache when an idle instruction mapping queue is found;

[0152] The writing module 906 is used to write each instruction set to be executed into different instruction queues corresponding to the instruction mapping queues in sequence; the instruction mapping queues are used to store the identifiers of the instruction sets to be executed and the identifiers of the instructions in the instruction sets to be executed;

[0153] The scheduling module 908 is used to schedule instructions in the pending instruction sets of different instruction queues to the execution units in parallel, so that each execution unit executes instructions in different pending instruction sets in parallel and executes instructions in the same pending instruction set in sequence.

[0154] In the above embodiment, by querying whether there is an instruction mapping queue in an idle state in the instruction mapping queue set, when an instruction mapping queue in an idle state is queried, it indicates that there is an instruction queue corresponding to the instruction mapping queue and in an idle state, so at least one to-be-executed instruction set can be read out from the instruction cache, and then the identification of the to-be-executed instruction set and the identification of the instructions in the to-be-executed instruction set are saved in the instruction mapping queue, and each to-be-executed instruction set is written into a different instruction queue in turn, so that the instruction queue that can be parallel can schedule instructions, the instructions between different instruction queues can be parallel and executed by the execution unit, and the instructions in the same instruction queue can be sequentially scheduled and executed by the execution unit, so that the execution efficiency of the instructions can be improved, and the probability of using the execution unit can be improved, thereby effectively improving the throughput and execution efficiency of the AI chip.

[0155] In one of the embodiments, the reading module 904 is further configured to sequentially read the instructions obtained by compiling the computer program from the memory;

[0156] The writing module 906 is further configured to write the read instructions into the corresponding to-be-executed instruction set in the instruction cache.

[0157] The reading module 904 is further configured to read the instructions in the to-be-executed instruction set from the instruction cache based on a first-in first-out manner or an instruction pointer manner, to obtain at least one to-be-executed instruction set; the to-be-executed instruction set matches the to-be-executed instruction set.

[0158] In one of the embodiments, as shown in Figure 10 The apparatus further comprises:

[0159] The querying module 902 is further configured to query a target instruction set that has been written into other instruction queues and has a dependency relationship with the to-be-executed instruction set.

[0160] The first obtaining module 910 is configured to obtain the execution state of each instruction in the target instruction set.

[0161] The reading module 904 is further configured to, if the execution state is an executed state, read the instructions in at least one to-be-executed instruction set from the instruction cache based on a first-in first-out manner or an instruction pointer manner.

[0162] In the above embodiment, when reading the to-be-executed instruction set from the instruction cache, it is needed to determine whether the target instruction set having the dependency relationship with the to-be-executed instruction set is executed, and only when the target instruction set is executed, the instruction in the to-be-executed instruction set is read from the instruction cache based on the first-in first-out manner or the instruction pointer manner. On the one hand, it can ensure that the to-be-executed instruction set can be executed smoothly, and on the other hand, it can avoid that other to-be-executed instruction sets cannot use the instruction set and the execution unit due to that the to-be-executed instruction set occupies the instruction queue and the execution unit, and can effectively improve the instruction execution efficiency.

[0163] In one of the embodiments, the query module 902 is further configured to: find the isolation instruction corresponding to the to-be-executed instruction set; read the target instruction set identifier from the isolation instruction; and in the other instruction queue, query the target instruction set identifier having the dependency relationship with the to-be-executed instruction set according to the target instruction set identifier.

[0164] In one of the embodiments, the first acquisition module 910 is further configured to: find the target instruction mapping queue in which the target instruction set identifier is stored in the instruction mapping queue set; and acquire the execution state of each instruction in the target instruction mapping queue.

[0165] In one of the embodiments, as shown in Figure 10 the device further comprises:

[0166] The compiling module 912 is configured to compile the computer program to obtain the corresponding instruction.

[0167] The writing module 906 is further configured to save the obtained instruction to the memory.

[0168] The reading module 904 is further configured to sequentially read the saved instruction from the memory when the processor is powered on or the preliminary data required for executing the obtained instruction is acquired.

[0169] In the above embodiment, when executing the instruction in the to-be-executed instruction set, the data necessary for executing the to-be-executed instruction set is acquired first, and then the instruction in the to-be-executed instruction set is executed. On the one hand, it can ensure that the instruction in the to-be-executed instruction set can be executed smoothly, and on the other hand, it can avoid that other to-be-executed instruction sets cannot use the instruction set and the execution unit due to that the to-be-executed instruction set occupies the instruction queue and the execution unit, and can effectively improve the instruction execution efficiency.

[0170] In one of the embodiments, as shown in Figure 10 the device further comprises:

[0171] The second obtaining module 914 is configured to obtain target data of the external environment and load the target data if the instructions stored in the memory depend on the prepared data when executed; or obtain target data generated by the executed instructions and load the target data; wherein the target data generated by the executed instructions and the target data stored in the external environment both belong to the prepared data.

[0172] In one of the embodiments, as shown in FIG. 1, the apparatus further includes: Figure 10

[0173] The first detection module 916 is configured to detect the execution state of the instructions in the instruction queue.

[0174] The first cleaning module 918 is configured to clear the target instruction in the instruction queue when it is determined that the target instruction in the instruction queue is executed according to the execution state; and delete the identification of the target instruction in the instruction mapping queue associated with the instruction queue.

[0175] In one of the embodiments, as shown in FIG. 1, the apparatus further includes: Figure 10

[0176] The second detection module 920 is configured to detect the execution state of the instructions in the instruction queue.

[0177] The marking module 922 is configured to mark the target instruction in the instruction mapping queue associated with the instruction queue as executed when it is determined that the target instruction in the instruction queue is executed according to the execution state.

[0178] The second cleaning module 924 is configured to clean the instruction mapping queue associated with the instruction queue when all the instructions in the instruction queue are executed; and clear all the instructions in the instruction queue.

[0179] In one of the embodiments, as shown in FIG. 1, the apparatus further includes: Figure 10

[0180] The adjusting module 926 is configured to set the execution state of the target instruction in the target instruction mapping queue as executed when the target instruction corresponding to the target instruction mapping queue in the occupied state in the instruction mapping queue set is executed.

[0181] The third cleaning module 928 is configured to clean the target instruction mapping queue when the execution state of each instruction in the target instruction mapping queue is executed.

[0182] In one of the embodiments, the instructions and the target instructions include coarse-grained instructions.

[0183] ​​​In the above embodiment, by deleting the executed instruction from the instruction mapping queue and deleting the executed instruction from the execution queue, the executed instruction can be prevented from occupying the cache space, and the effective utilization rate of the cache space can be improved.

[0184] The specific limitation of the instruction scheduling apparatus can refer to the limitation of the instruction scheduling method in the above, which will not be described here. Each module in the above instruction scheduling apparatus can be realized by software, hardware and combination thereof in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operation corresponding to each module.

[0185] In one embodiment, a computer device is provided, which can be a terminal, and the internal structure diagram thereof can be as shown in Figure 11 The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be achieved by WIFI, operator network, NFC (near field communication) or other technologies. The computer program is executed by the processor to implement an instruction scheduling method. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the computer device, or an external keyboard, touchpad or mouse, etc.

[0186] Those skilled in the art can understand that Figure 11 the structure shown in the above is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0187] In one embodiment, as shown in Figure 3 an artificial intelligence chip is provided, which includes a state query unit, an instruction scheduling unit and an execution unit; the instruction scheduling unit includes an instruction cache, an instruction gating subunit, an instruction distribution subunit and an instruction queue unit;

[0188] The state query unit is configured to query an instruction mapping queue in an idle state in the instruction mapping queue set.

[0189] The instruction cache is configured to store the to-be-executed instruction set.

[0190] The instruction gating subunit is configured to read the to-be-executed instruction set from the instruction cache when the instruction mapping queue in the idle state is queried.

[0191] The instruction distribution subunit is configured to sequentially write the read to-be-executed instruction set into different instruction queues corresponding to the instruction mapping queue.

[0192] The instruction queue unit is configured to schedule the instructions in the to-be-executed instruction set in different instruction queues to the execution unit in a parallel manner.

[0193] The execution unit is configured to execute the instructions in different to-be-executed instruction sets in parallel and sequentially execute the instructions in the same to-be-executed instruction set according to the instruction order.

[0194] In the above embodiment, whether there is an instruction mapping queue in an idle state in the instruction mapping queue set is queried. When the instruction mapping queue in the idle state is queried, it indicates that there is an instruction queue corresponding to the instruction mapping queue and in the idle state. Therefore, at least one to-be-executed instruction set can be read from the instruction cache, the identification of the to-be-executed instruction set and the identification of the instructions in the to-be-executed instruction set are saved in the instruction mapping queue, and each to-be-executed instruction set is sequentially written into different instruction queues. Therefore, there are instruction queues that can be scheduled in parallel, the instructions between different instruction queues can be scheduled in parallel and executed by the execution unit, the instructions in the same instruction queue can be sequentially scheduled and executed by the execution unit, the execution efficiency of the instructions can be improved, the probability of using the execution unit can be improved, and the throughput and execution efficiency of the AI chip are effectively improved.

[0195] In one embodiment, the instruction scheduling unit is further configured to sequentially read the instructions obtained by compiling the computer program from the memory, write the read instructions into the corresponding to-be-executed instruction set in the instruction cache, and read the instructions in the to-be-executed instruction set from the instruction cache based on a first-in-first-out manner or an instruction pointer manner through the instruction gating subunit.

[0196] In one embodiment, the instruction distribution subunit is further configured to query a target instruction set that has been written into other instruction queues and has a dependency relationship with the to-be-executed instruction set.

[0197] The state query unit is further configured to obtain the execution state of each instruction in the target instruction set.

[0198] The instruction gating subunit is further configured to read out the instructions in the to-be-executed instruction set from the instruction cache based on a first-in first-out manner or an instruction pointer manner if the execution state is the executed state.

[0199] In one embodiment, the instruction distribution subunit is further configured to search for an isolated instruction corresponding to the to-be-executed instruction set, read a target instruction set identifier from the isolated instruction, and search for a target instruction set identifier having a dependency relationship with the to-be-executed instruction set in the other instruction queue according to the target instruction set identifier.

[0200] In one embodiment, as shown in FIG. 6, the state query unit includes a queue state query subunit and an instruction queue mapping subunit. Figure 3

[0201] The queue state query subunit is further configured to search for a target instruction mapping queue storing the target instruction set identifier in the instruction mapping queue set.

[0202] The instruction queue mapping subunit is configured to acquire the execution state of each instruction in the target instruction mapping queue.

[0203] In the above embodiment, when the to-be-executed instruction set is read from the instruction cache, it is necessary to determine whether the target instruction set having a dependency relationship with the to-be-executed instruction set is executed. Only when the target instruction set is executed, the instructions in the to-be-executed instruction set are read out from the instruction cache based on the first-in first-out manner or the instruction pointer manner. On the one hand, it can ensure that the to-be-executed instruction set read out can be executed smoothly. On the other hand, it can avoid that other to-be-executed instruction sets cannot use the instruction set and the execution unit due to that the to-be-executed instruction set occupies the instruction queue and the execution unit, thereby effectively improving the instruction execution efficiency.

[0204] In one embodiment, the artificial intelligence chip further includes a compiling unit.

[0205] The compiling unit is configured to compile a computer program to obtain corresponding instructions, and save the obtained instructions to a memory.

[0206] The instruction scheduling unit is further configured to sequentially read the saved instructions from the memory when the processor is powered on or when the prepared data required for executing the obtained instructions is acquired.

[0207] In one embodiment, the instruction scheduling unit is further configured to acquire and load the prepared data from the memory or the target cache if the instructions saved in the memory depend on the prepared data when executed; the prepared data includes data generated by the executed instructions.

[0208] ​In the above embodiments, when executing the instructions in the to-be-executed instruction set, the data necessary for executing the to-be-executed instruction is acquired first, and then the instructions in the to-be-executed instruction set are executed, which can ensure that the instructions in the to-be-executed instruction set can be executed smoothly, and can avoid that other to-be-executed instruction sets cannot use the instruction set and the execution unit due to that the to-be-executed instruction set occupies the instruction queue and the execution unit, and can effectively improve the instruction execution efficiency.

[0209] In one embodiment, the queue state querying sub-unit is configured to detect the execution state of the instructions in the instruction queue.

[0210] The instruction queue sub-unit is further configured to clear the target instruction in the instruction queue when it is determined according to the execution state that the target instruction in the instruction queue is executed.

[0211] The instruction queue mapping sub-unit is further configured to delete the identification of the target instruction in the instruction mapping queue associated with the instruction queue.

[0212] In one embodiment, the queue state querying sub-unit is configured to detect the execution state of the instructions in the instruction queue.

[0213] When it is determined according to the execution state that the target instruction in the instruction queue is executed, the identification of the target instruction in the instruction mapping queue associated with the instruction queue is marked as executed.

[0214] The instruction queue mapping sub-unit is further configured to clean the instruction mapping queue associated with the instruction queue when all the instructions in the instruction queue are executed.

[0215] The instruction queue sub-unit is further configured to clear all the instructions in the instruction queue.

[0216] In one embodiment, the instructions and the target instruction include coarse-grained instructions.

[0217] In the above embodiments, by deleting the identification of the executed instructions from the instruction mapping queue and deleting the executed instructions from the execution queue, the occupied cache space by the executed instructions can be avoided, and the effective utilization rate of the cache space can be improved.

[0218] In one embodiment, a computer device is also provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0219] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps in the above method embodiments when executed by a processor.

[0220] In an embodiment, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in each of the above method embodiments.

[0221] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiments. Any reference to memory, storage, database or other medium used in each embodiment provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0222] Each of the technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, not all possible combinations of each technical feature in the above embodiments are described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0223] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, some modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of the patent protection of the present application should be subject to the appended claims.

Claims

1. An instruction scheduling method, characterized in that: The method comprises: sequentially reading instructions obtained by compiling a computer program corresponding to a target task from a memory; Writing the read instruction into the corresponding set of instructions to be executed in the instruction cache; wherein the target task is divided into multiple subtasks according to the granularity, and each set of instructions to be executed stores the instructions of the corresponding subtask; the multiple subtasks include coarse-grained tasks; Query the idle instruction mapping queue in the instruction mapping queue set; When an idle instruction mapping queue is found, a target instruction set that has been written into other instruction queues and has a dependency relationship with the instruction set to be executed is searched; the execution status of each instruction in the target instruction set is obtained; if the execution status is an executed state, the instructions in the instruction set to be executed are read from the instruction cache based on a first-in-first-out method or an instruction pointer method; writing the read instruction sets to be executed into different instruction queues corresponding to the instruction mapping queues in sequence; the instruction mapping queues are used to store the identifiers of the instruction sets to be executed and the identifiers of the instructions in the instruction sets to be executed; Instructions in the instruction sets to be executed of different instruction queues are dispatched to the execution units in a parallel manner, so that each execution unit executes the instructions in different instruction sets to be executed in parallel, and executes the instructions in the same instruction set to be executed sequentially according to the instruction order; each execution unit includes execution units with multiple functions, and there are multiple execution units for each function.

2. The method according to claim 1, characterized in that The query includes target instruction sets that have been written into other instruction queues and have a dependency relationship with the instruction set to be executed: Searching for isolated instructions corresponding to the set of instructions to be executed; Reading a target instruction set identifier from the isolated instruction; In the other instruction queues, a target instruction set identifier that has a dependency relationship with the instruction set to be executed is searched according to the target instruction set identifier.

3. The method according to claim 2, characterized in that The acquiring the execution status of each instruction in the target instruction set includes: Searching the target instruction mapping queue storing the target instruction set identifier in the instruction mapping queue set; The execution status of each instruction is obtained in the target instruction mapping queue.

4. The method according to claim 1, wherein The method further comprises: Compile the computer program to obtain the corresponding instructions; saving the obtained instruction to the memory; The instructions obtained by sequentially reading the compiled computer program from the memory include: When the processor is powered on, or after obtaining the preparation data required for executing the obtained instructions, the stored instructions are read sequentially from the memory.

5. The method according to claim 4, characterized in that The method further comprises: If the instructions stored in the memory depend on the prepared data when they are executed, then The preparation data is obtained from the memory or the target cache and loaded; the preparation data includes data generated by the executed instructions.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: detecting the execution status of instructions in the instruction queue; When it is determined according to the execution status that the target instruction in the instruction queue is completed, clearing the target instruction in the instruction queue; and In the instruction mapping queue associated with the instruction queue, the identifier of the target instruction is deleted.

7. The method according to any one of claims 1 to 5, characterized in that The method further comprises: detecting the execution status of instructions in the instruction queue; When it is determined according to the execution status that the target instruction in the instruction queue is completed, marking the target instruction as being in an executed state in an instruction mapping queue associated with the instruction queue; When all instructions in the instruction queue are executed, the instruction mapping queue associated with the instruction queue is cleared; and Clear all instructions in the instruction queue.

8. The method according to any one of claims 1 to 5, characterized in that The instructions and target instructions include coarse-grained instructions; the target instructions include executed instructions in an instruction queue.

9. An artificial intelligence chip, characterized in that: The artificial intelligence chip includes: a status query unit, an instruction scheduling unit and an execution unit; the instruction scheduling unit includes an instruction cache, an instruction gating subunit, an instruction distribution subunit and an instruction queue unit; The instruction scheduling unit is configured to sequentially read instructions obtained by compiling a computer program corresponding to a target task from a memory; and write the read instructions into a corresponding set of instructions to be executed in the instruction cache; wherein the target task is divided into a plurality of subtasks according to granularity, and each set of instructions to be executed stores instructions corresponding to a subtask; and the plurality of subtasks include coarse-grained tasks; The state query unit is used to query the instruction mapping queue in the instruction mapping queue set for an idle state; The instruction gating subunit is configured to, when an idle instruction mapping queue is queried, query a target instruction set that has been written into other instruction queues and has a dependency relationship with the instruction set to be executed; obtain the execution status of each instruction in the target instruction set; and if the execution status is an executed state, read the instructions in the instruction set to be executed from the instruction cache based on a first-in-first-out method or an instruction pointer method; The instruction dispatch subunit is used to write the read instruction sets to be executed into different instruction queues corresponding to the instruction mapping queues in sequence; the instruction mapping queues are used to store the identifiers of the instruction sets to be executed and the identifiers of the instructions in the instruction sets to be executed; The instruction queue unit is used to dispatch instructions in the to-be-executed instruction sets of different instruction queues to the execution unit in a parallel manner; The execution unit is used to enable each execution unit to execute instructions in different instruction sets to be executed in parallel, and to execute instructions in the same instruction set to be executed sequentially according to the instruction order; each execution unit includes an execution unit with multiple functions, and there are multiple execution units for each function.

10. The artificial intelligence chip according to claim 9, characterized in that: The instruction gating subunit is further used to search for an isolated instruction corresponding to the instruction set to be executed; read a target instruction set identifier from the isolated instruction; and query the target instruction set identifier that has a dependency relationship with the instruction set to be executed in the other instruction queue according to the target instruction set identifier.

11. The artificial intelligence chip according to claim 10, characterized in that: The status query unit is further configured to search the instruction mapping queue set for a target instruction mapping queue storing the target instruction set identifier; and obtain the execution status of each instruction in the target instruction mapping queue.

12. The artificial intelligence chip according to claim 9, characterized in that: The instruction scheduling unit is also used to compile a computer program to obtain corresponding instructions; save the obtained instructions to the memory; and read the saved instructions sequentially from the memory when the processor is powered on or after obtaining the preliminary data required to execute the obtained instructions.

13. The artificial intelligence chip according to claim 12, characterized in that: The instruction scheduling unit is further configured to obtain and load the prepared data from the memory or target cache if the instruction stored in the memory depends on the prepared data during execution; the prepared data includes data generated by the executed instruction.

14. The artificial intelligence chip according to any one of claims 9 to 13, characterized in that: The status query unit is also used to detect the execution status of the instructions in the instruction queue; when it is determined that the execution of the target instruction in the instruction queue is completed according to the execution status, the instruction queue unit is also used to clear the target instruction in the instruction queue; the status query unit is also used to delete the identifier of the target instruction in the instruction mapping queue associated with the instruction queue.

15. The artificial intelligence chip according to any one of claims 9 to 13, characterized in that: The status query unit is also used to detect the execution status of the instructions in the instruction queue; when it is determined that the target instruction in the instruction queue is completed according to the execution status, the target instruction is marked as executed in the instruction mapping queue associated with the instruction queue; when all instructions in the instruction queue are completed, the status query unit is also used to clear the instruction mapping queue associated with the instruction queue; the instruction queue unit is also used to clear all instructions in the instruction queue.

16. The artificial intelligence chip according to any one of claims 9 to 13, characterized in that: The instructions and target instructions include coarse-grained instructions; the target instructions include executed instructions in an instruction queue.

17. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

18. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

19. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images