Method for distributing compute core instructions, artificial intelligence chip, computing device, medium and program product

Abstract

This invention relates to a method for allocating computational core instructions, an artificial intelligence chip, a computing device, a medium, and a program product, relating to the field of artificial intelligence technology. The method includes: selecting from multiple instruction channels associated with computational core instructions to be allocated; caching the computational core instructions from the selected instruction channels into a computational core instruction cache unit corresponding to the computational core to which the instructions are destined; selecting a non-empty computational core instruction cache unit, and reading the computational core instructions from the selected cache unit and sending them to the corresponding computational core, so that instruction channel selection and computational core instruction sending are performed in parallel. This invention can significantly improve the efficiency of computational core instruction sending, thereby improving the actual computing performance of the artificial intelligence chip.

Description

Technical Field

[0001] The embodiments of the present invention generally relate to the field of artificial intelligence technology, and more specifically to a method for allocating computing core instructions, an artificial intelligence chip, a computing device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] In traditional methods for allocating computational core instructions, during multi-channel command allocation, channel selection is required after commands are issued from different channels to choose one for command distribution. Taking the video encoding / decoding module in an AI chip as an example, a traditional general-purpose computing on graphics processing units (GPGPU) processes data used for training and inference of large models. The video encoding / decoding module (e.g., video codec) is responsible for encoding and decoding the data to be trained and then feeding the decoded raw data into the computational core for training. The process of feeding raw data into the computational core is implemented based on a task distribution unit (e.g., vcpd) and a task execution unit (e.g., video core). The task distribution unit distributes video-related computational core instructions to various task execution units, allowing the task execution units to parse and execute the video encoding / decoding operations. In the multi-channel computational core instruction allocation process, typically only one instruction channel is selected for distribution, and the computational core instructions are distributed to the corresponding computational core through the instructions within the selected channel. During instruction distribution, instructions from other instruction channels cannot be received, thus significantly reducing the efficiency of computational core instruction distribution.

[0003] In summary, the shortcomings of traditional methods for allocating computational core instructions are that they cannot receive computational core instructions from other instruction channels during the process of issuing multi-channel computational core instructions, thereby reducing the efficiency of computational core instruction issuance and restricting the actual computing performance of artificial intelligence chips. Summary of the Invention

[0004] This invention provides a method for allocating computation core instructions, an artificial intelligence chip, a computing device, a computer-readable storage medium, and a computer program product, which can significantly improve the efficiency of computation core instruction issuance, thereby improving the actual computing performance of the artificial intelligence chip.

[0005] According to a first aspect of the present invention, a method for allocating computation core instructions is provided, the method comprising: selecting a plurality of instruction channels associated with computation core instructions to be allocated; caching the computation core instructions in the selected instruction channels into a computation core instruction cache unit corresponding to the computation core to which the computation core instructions are to be allocated; selecting a computation core instruction cache unit that is in a non-empty state, and reading out the computation core instructions in the selected computation core instruction cache unit and sending them to the corresponding computation core, so that instruction channel selection and computation core instruction sending are carried out in parallel.

[0006] In some embodiments, each of the multiple instruction channels is configured with its own instruction allocation control state machine, and the computation core instructions are used for multiple computation cores of the video encoding and decoding unit of the artificial intelligence chip.

[0007] In some embodiments, in response to determining that the computation core instruction of the current instruction channel is valid, the instruction allocation control state machine enters the pre-allocation state; in response to determining that the computation core instruction cache unit corresponding to the computation core to which the computation core instruction of the current instruction channel is to be sent is not full, the instruction allocation control state machine enters the candidate allocation state; and in response to determining that the current instruction channel is selected, the instruction allocation control state machine enters the idle state.

[0008] In some embodiments, caching the core instruction in the selected instruction channel to the core instruction cache unit corresponding to the core to which the core instruction is to be cached includes: caching the current core instruction packet in the selected current instruction channel to the core instruction cache unit corresponding to the core to which the current core instruction packet is to be cached; and in response to determining that the current core instruction packet in the current instruction channel has been cached, selecting the next instruction channel so that the next core instruction packet in the selected next instruction channel is cached to the core instruction cache unit corresponding to the core to which the next core instruction packet is to be cached.

[0009] In some embodiments, selecting from multiple instruction channels associated with a compute core instruction to be allocated includes: in response to determining that the compute core instruction cache unit corresponding to the compute core to which the compute core instruction of the current instruction channel is to be sent is not full, determining the current instruction channel as a candidate instruction channel, and the instruction allocation control state machine configured for the candidate instruction channel entering a candidate allocation state; and randomly selecting from all candidate instruction channels to determine the selected instruction channel.

[0010] In some embodiments, selecting a non-empty core instruction cache unit and reading the core instructions from the selected core instruction cache unit and sending them to the corresponding core includes: in response to determining that the current core instruction cache unit is in a non-empty state and the corresponding core is in an idle state, determining the current core instruction cache unit as a candidate state; randomly selecting from all candidate core instruction cache units via a core cache unit selection module; and reading the core instructions from the selected core instruction cache unit and sending them to the corresponding core.

[0011] In some embodiments, in response to determining that the computational core instruction of the current instruction channel is valid, the instruction allocation control state machine enters the pre-allocation state, which includes: each instruction allocation control state machine determining whether there is a computational core instruction in the corresponding instruction channel; and in response to determining that there is a computational core instruction in the corresponding instruction channel, the instruction allocation control state machine enters the pre-allocation state.

[0012] In some embodiments, the instruction allocation control state machine configured for the candidate instruction channel entering the candidate allocation state includes: each instruction allocation control state machine checking whether the computational core instruction cache unit corresponding to the computational core to which the computational core instruction in the corresponding instruction channel is to be sent is not in a full state; in response to determining that the corresponding computational core instruction cache unit is not in a full state, the instruction allocation control state machine enters the candidate allocation state; and generates a valid candidate allocation identifier.

[0013] In some embodiments, caching the core instructions in the selected instruction channel to the core instruction cache unit corresponding to the core to which the core instructions are to be sent includes: the instruction channel selection module randomly selecting an instruction channel corresponding to the instruction allocation control state machine in the candidate allocation state based on the candidate allocation valid identifier; and caching the core instructions in the selected instruction channel to the core instruction cache unit corresponding to the core to which the core instructions are to be sent.

[0014] According to a second aspect of the present invention, an artificial intelligence chip is also provided, the artificial intelligence chip comprising: a plurality of instruction channels configured to allocate computing core instructions; an instruction channel selection module configured to select from the plurality of instruction channels where the computing core instructions are located; a computing core instruction cache unit corresponding to a computing core, the computing core instruction cache unit being configured to cache the computing core instructions in the selected instruction channel to the computing core instruction cache unit corresponding to the computing core to which the computing core instructions are to be sent; and a computing core cache unit selection module configured to select a computing core instruction cache unit that is in a non-empty state, and read out the computing core instructions in the selected computing core instruction cache unit and send them to the corresponding computing core.

[0015] In some embodiments, multiple instruction allocation control state machines are associated with multiple instruction channels respectively. The instruction allocation control state machines are configured to: enter a pre-allocation state in response to determining that the computation core instruction of the current instruction channel is valid; enter a candidate allocation state in response to determining that the computation core instruction cache unit corresponding to the computation core to which the computation core instruction of the current instruction channel is to be sent is not full; and enter an idle state in response to determining that the current instruction channel is selected.

[0016] According to a third aspect of the present invention, a computing device is also provided. The computing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the computing device to perform the method of the first aspect of the present invention.

[0017] According to a fourth aspect of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0018] According to a fifth aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a machine, performs the method of the first aspect of the present invention.

[0019] This invention caches the core instructions from the selected instruction channel into the core instruction cache unit corresponding to the core to which the core instructions are destined; selects a non-empty core instruction cache unit, reads the core instructions from the selected cache unit, and sends them to the corresponding core. This allows instruction channel selection and core instruction sending to proceed in parallel. The invention synchronizes the core instruction allocation process of the instruction channel with the process of sending core instructions from the core instruction cache unit to the corresponding core, thereby improving the efficiency of core instruction sending and ultimately enhancing the actual computing performance of the AI ​​chip. Furthermore, this invention configures each instruction channel with its own instruction allocation control state machine, allowing for independent allocation between different channels without synchronization, further improving the efficiency of core instruction sending.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0022] Figure 1 The diagram illustrates a computing apparatus for implementing a method of allocating computing core instructions according to an embodiment of the present invention.

[0023] Figure 2 A flowchart of a method for allocating computation kernel instructions according to some embodiments of the present invention is shown.

[0024] Figure 3 A schematic diagram of the structure of an artificial intelligence chip according to some embodiments of the present invention is shown.

[0025] Figure 4 A flowchart of a method for allocating computation kernel instructions according to some embodiments of the present invention is shown.

[0026] Figure 5 A flowchart of a method for allocating computation kernel instructions according to some embodiments of the present invention is shown.

[0027] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0028] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0029] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0030] As described above, the traditional method for allocating computing core instructions has the following drawbacks: it cannot receive computing core instructions from other instruction channels during the process of issuing computing core instructions for multi-channel computing cores, thereby reducing the efficiency of computing core instruction issuance and restricting the actual computing performance of artificial intelligence chips.

[0031] To at least partially address one or more of the aforementioned problems and other potential issues, an exemplary embodiment of the present invention proposes a method for allocating computational core instructions. In this method, multiple instruction channels associated with the computational core instructions to be allocated are selected; the computational core instructions from the selected instruction channels are cached in the computational core instruction cache unit corresponding to the computational core to which the computational core instructions are destined; a non-empty computational core instruction cache unit is selected, and the computational core instructions in the selected computational core instruction cache unit are read out and sent to the corresponding computational core, so that instruction channel selection and computational core instruction sending are parallelized. Since the selected instruction channel caches the computational core instructions in the channel in the computational core instruction cache unit corresponding to the computational core to which the computational core instructions are destined, the selection and allocation of instruction channels for the computational core instructions to be allocated are continuously performed simultaneously with the sending of computational core instructions to the computational core. That is, the process of allocating computational core instructions from the instruction channel and the process of sending computational core instructions from the computational core instruction cache unit to the corresponding computational core do not need to wait for each other. Therefore, the present invention enables the parallelization of the allocation and sending processes of computational core instructions, thereby improving the efficiency of computational core instruction sending and thus improving the actual computing performance of the artificial intelligence chip.

[0032] Figure 1 A schematic diagram of a computing device 100 for implementing a method of allocating computing core instructions according to an embodiment of the present invention is shown. Figure 1 As shown, the computing device 100 may have one or more processing units and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. The processing unit includes dedicated processing units such as graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), and general-purpose computing on graphics processing units (GPGPUs), as well as general-purpose processing units such as CPUs. The computing device 100 also includes at least: an instruction channel selection unit 102 associated with computing core instructions, a computing core instruction cache unit 104 corresponding to the computing core, and a computing core instruction read and write unit 106.

[0033] Regarding the instruction channel selection unit 102 associated with the computing core instruction, it is used to select from multiple instruction channels associated with the computing core instruction to be assigned.

[0034] Regarding the computation core instruction cache unit 104 corresponding to the computation core, it is used to cache the computation core instructions in the selected instruction channel to the computation core instruction cache unit corresponding to the computation core to which the computation core instructions are to be sent.

[0035] Regarding the core instruction read and send unit 106, it is used to select a core instruction cache unit that is not empty, and read the core instructions in the selected core instruction cache unit and send them to the corresponding core, so that the instruction channel selection and core instruction sending can be carried out in parallel.

[0036] The following will combine Figure 2 and Figure 3 A method 200 for allocating computation kernel instructions, according to embodiments of the present invention, is described. It should be understood that method 200 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 200 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0037] At step 202, the computing device 100 selects from a plurality of instruction channels associated with the computing core instruction to be assigned.

[0038] In some embodiments, the computational core instructions are, for example, used to command multiple computational cores of a video codec unit of an artificial intelligence chip. Specifically, the video codec unit is used, for example, to encode and decode video data used for training and inference of large models.

[0039] Regarding the instruction channel, it is configured to allocate computational core instructions. It should be understood that an AI chip includes multiple instruction channels (such as...). Figure 3 As indicated by mark 302), each of the multiple instruction channels is configured with its own instruction allocation control state machine (e.g., Figure 3 (As indicated by mark 304). Figure 3 For example, eight instruction channels are shown. These include instruction channels 0 through 7. Each instruction channel is configured with its own instruction allocation control state machine. For example, instruction channels 0 through 7 are each configured with instruction allocation control state machines 0 through 7.

[0040] For each instruction allocation control state machine, its state may include, for example, a pre-allocation state, a candidate allocation state, and an idle state.

[0041] In some embodiments, a method for selecting from multiple instruction channels includes, for example,: if the computing device 100 determines that the instruction cache unit corresponding to the computing core to which the current instruction channel's core instruction is to be sent is not full, determining the current instruction channel as a candidate instruction channel, and the instruction allocation control state machine configured for the candidate instruction channel entering a candidate allocation state; and randomly selecting from all candidate instruction channels to determine the selected instruction channel. In some embodiments, via Figure 3 The instruction channel selection module 306 shown randomly selects the instruction channel corresponding to the instruction allocation control state machine that is in the candidate allocation state. For example, if instruction allocation control state machine 2, instruction allocation control state machine 3, and instruction allocation control state machine 7 are in the candidate allocation state, the instruction channel selection module 306 randomly selects the corresponding instruction channel 2, instruction channel 3, and instruction channel 7. The following will combine... Figure 4 The specific methods for selecting multiple instruction channels will not be elaborated here.

[0042] In step 204, the computing device 100 caches the core instructions in the selected instruction channel into the core instruction cache unit corresponding to the core to which the core instructions are to be sent.

[0043] In some embodiments, the computation core instruction cache unit may be, for example, a separate computation core instruction cache unit configured for each computation core, which is used to cache computation core instructions via a selected instruction channel and to distribute the cached computation core instructions to the corresponding computation core. Figure 3 As shown, artificial intelligence chip ( Figure 3 (Not shown) For example, it includes four computing cores, namely computing core 0 to computing core 3. Each computing core is configured with its own computing core instruction cache unit, such as... Figure 3 As shown, computing cores 0 to 3 are respectively configured with computing core instruction cache units 0 to 3.

[0044] The method for caching core instructions in a selected instruction channel to a corresponding core instruction cache unit includes, for example, caching the current core instruction packet in the selected current instruction channel to the core instruction cache unit corresponding to the core to which the current core instruction packet is destined; and, in response to determining that the current core instruction packet in the current instruction channel has been completely cached, selecting the next instruction channel so that the next core instruction packet in the selected next instruction channel is cached to the core instruction cache unit corresponding to the core to which the next core instruction packet is destined. For example, the computing device 100 caches the core instruction packet in the selected instruction channel 2 to the core instruction cache unit 0 corresponding to the core 0 to which the current core instruction packet is destined. If it is determined that the core instruction packet in instruction channel 2 has been completely cached, the computing device 100 selects the next instruction channel. For example, if instruction channel 7 is selected, the computing device 100 caches the core instruction packet in instruction channel 7 to the core instruction cache unit 2 corresponding to core 2. By employing the above means, the present invention can continuously select and cache instruction channels.

[0045] In step 206, the computing device 100 selects a computing core instruction cache unit that is in a non-empty state, and reads out the computing core instructions in the selected computing core instruction cache unit and sends them to the corresponding computing core, so that the instruction channel selection and computing core instruction sending are carried out in parallel.

[0046] A method for reading and sending core instructions from a selected core instruction cache unit to the corresponding core may include, for example, the following steps: if the computing device 100 determines that the current core instruction cache unit is not empty and the corresponding core is idle, determining the current core instruction cache unit as a candidate; randomly selecting from all candidate core instruction cache units via a core cache unit selection module; and reading and sending the core instructions from the selected core instruction cache unit to the corresponding core. For example, if it is determined that... Figure 3 If the core instruction cache units 0 and 2 shown are in a non-empty state, and the corresponding cores 0 and 2 are in an idle state, then core instruction cache units 0 and 2 are determined to be candidate states. Then, via the core instruction cache unit selection module 310, a random selection is performed on all candidate core instruction cache units 0 and 2; for example, core instruction cache unit 2 is ultimately selected. The computing device 100 reads the core instructions from the selected core instruction cache unit 2 and sends them to the corresponding core 2.

[0047] In the above scheme, by caching the computational core instructions in the selected instruction channel to the computational core instruction cache unit corresponding to the computational core to which the computational core instructions are to be sent, and by selecting a non-empty computational core instruction cache unit, reading out the computational core instructions in the selected computational core instruction cache unit and sending them to the corresponding computational core, the present invention enables the parallel execution of the instruction channel selection and the computational core instruction sending process. This allows the computational core instruction allocation process of the instruction channel to be synchronized with the process of sending the computational core instructions from the computational core instruction cache unit to the corresponding computational core. Therefore, the present invention enables the allocation and sending process of computational core instructions to be parallelized, thereby significantly improving the efficiency of computational core instruction sending and thus improving the actual computing performance of the artificial intelligence chip.

[0048] The following combination Figure 3 Provide a detailed description of the structure of an artificial intelligence chip. For example... Figure 3 As shown, the artificial intelligence chip includes, for example, multiple instruction channels 302, an instruction channel selection module 306, multiple computing core instruction cache units 308, a computing core cache unit selection module 310, and multiple computing cores 312. In some embodiments, the artificial intelligence chip further includes, for example, multiple instruction allocation control state machines 304.

[0049] Regarding multiple instruction channels 302, they are configured, for example, to allocate computational core instructions. For example... Figure 3 As shown, the multiple instruction channels 302 include, for example, instruction channel 0, instruction channel 1 to instruction channel 7. In some embodiments, each instruction channel 302 is independent of each other, each is configured with an independent dispatch mechanism, and each independently receives the computation kernel instructions of the computation task to be executed.

[0050] Regarding the multiple instruction allocation control state machine 304, it is configured, for example, such that each instruction channel in the multiple instruction channels is configured with its own instruction allocation control state machine. For example... Figure 3As shown, the plurality of instruction allocation control state machines 304 may include, for example, instruction allocation control state machine 0, instruction allocation control state machines 1 to 7. In some embodiments, each instruction channel is connected to an instruction allocation control state machine for controlling the instruction allocation state of that instruction channel. For example, the instruction allocation operation of instruction channel 0 is controlled by instruction allocation control state machine 0. The states of each instruction allocation control state machine may include, for example, a pre-allocation state, a candidate allocation state, and an idle state. The configuration method for the state of the instruction allocation control state machine may include, for example, the following: in response to determining that the core instruction of the current instruction channel is valid, the instruction allocation control state machine enters the pre-allocation state; in response to determining that the core instruction cache unit corresponding to the core to which the core instruction of the current instruction channel is to be sent is not full, the instruction allocation control state machine enters the candidate allocation state; and in response to determining that the current instruction channel is selected, the instruction allocation control state machine enters the idle state.

[0051] Regarding the instruction channel selection module 306, it is configured, for example, to select from multiple instruction channels where the computation core instructions reside. Figure 3 As shown, the instruction channel selection module 306 is electromechanically connected to multiple instruction channels 302 through their respective instruction allocation control states, and stores the core instructions in the selected instruction channel into the core instruction cache unit corresponding to the core to which the core instructions are to be executed.

[0052] Regarding the multiple compute core instruction cache units 308, they are configured, for example, to correspond to a compute core. For example... Figure 3 As shown, computing cores 0 to 3 correspond to computing core instruction cache units 0 to 3. The computing core instruction cache unit is configured to cache the computing core instructions from the selected instruction channel into the computing core instruction cache unit corresponding to the computing core to which the instruction is destined. For example, if it is determined that the computing core instruction cache unit 0 corresponding to computing core 0, to which the instruction from instruction channel 2 is destined, is not full, instruction channel 2 is determined as a candidate instruction channel, and the instruction allocation control state machine configured for the candidate instruction channel enters the candidate allocation state; the instruction channel selection module 306 selects from the candidate instruction channel 2 to determine the selected instruction channel. The computing core instructions from the selected instruction channel 2 are cached into the computing core instruction cache unit 0 corresponding to the computing core 0 to which the instruction is destined.

[0053] Regarding the core cache unit selection module 310, it is configured, for example, to: select a core instruction cache unit that is not empty, and read the core instructions from the selected core instruction cache unit and send them to the corresponding core. For example, if it is determined that core instruction cache unit 0 and core instruction cache unit 2 are not empty, and the corresponding cores 0 and 2 are idle, core instruction cache unit 0 and core instruction cache unit 2 are determined as candidate states; the core cache unit selection module 310 randomly selects from the candidate core instruction cache unit 0 and core instruction cache unit 2, for example, selecting core instruction cache unit 2; and reads the core instructions from the selected core instruction cache unit 2 and sends them to the corresponding core.

[0054] Regarding multiple computing cores 312, they are configured, for example, to execute corresponding computing tasks based on computing core instructions read and issued by the computing core cache unit selection module 310. For example... Figure 3 As shown, the plurality of computing cores 312 include, for example, computing core 0, computing core 1, computing core 2, and computing core 3. In some embodiments, each computing core can independently perform computing tasks.

[0055] In the above scheme, a core instruction cache unit is set up between the instruction channel selection module and the core instruction and core cache unit selection module to cache core instructions. When an instruction channel is selected, the core instruction is not immediately sent to the corresponding core. Instead, it is first cached in the core instruction cache unit corresponding to the core. After caching, the core instruction of the next instruction channel is immediately selected and stored in a non-empty core instruction cache unit. At the same time, the core instruction cache unit sends the core instruction to the corresponding core. Thus, this invention can ensure that the selection of instruction channels and the sending of core instructions can be carried out in parallel. This pipelined operation greatly improves the efficiency of core instruction allocation and sending.

[0056] The following will combine Figure 3 and Figure 4 The present invention describes a method 400 for allocating computation kernel instructions, according to an embodiment of the invention. It should be understood that method 400 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 400 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0057] At step 402, in response to determining that the compute core instruction cache unit corresponding to the compute core to which the compute core instruction of the current instruction channel is to be sent is not full, the computing device 100 determines that the current instruction channel is a candidate instruction channel, and the instruction allocation control state machine configured for the candidate instruction channel enters the candidate allocation state.

[0058] In some embodiments, the method for the instruction allocation control state machine configured for the candidate instruction channel to enter the candidate allocation state includes, for example: each instruction allocation control state machine checks (as shown by identifier 314) whether the computational core instruction cache unit corresponding to the computational core to which the computational core instruction in the corresponding instruction channel is to be sent is in a non-full state; in response to determining that the corresponding computational core instruction cache unit is in a non-full state, the instruction allocation control state machine enters the candidate allocation state; and generates an identifier indicating that the candidate allocation is valid.

[0059] The fact that a compute core instruction cache unit is not full indicates, for example, that the cache space in the compute core instruction cache unit has not yet reached its capacity limit and can continue to be used to cache new compute core instructions.

[0060] At step 404, the computing device 100 randomly selects from all candidate instruction channels in order to determine the selected instruction channel.

[0061] In some embodiments, the method for caching the core instructions in the selected instruction channel to the core instruction cache unit corresponding to the core to which the core instructions are to be sent includes, for example,: the instruction channel selection module 306 randomly selects the instruction channel corresponding to the instruction allocation control state machine in the candidate allocation state based on the candidate allocation valid identifier; and caches the core instructions in the selected instruction channel to the core instruction cache unit corresponding to the core to which the core instructions are to be sent.

[0062] Regarding the identifier of a valid candidate allocation, such as pipe_vld, for example... Figure 3 As shown, for example, the valid candidate allocation identifier for instruction allocation control state machine 0 is pipe_vld0. The valid candidate allocation identifier for instruction allocation control state machine 1 is pipe_vld1, and so on. The method for generating the valid candidate allocation identifier is, for example, as follows: each instruction allocation control state machine checks whether the instruction cache unit corresponding to the core to which the instruction in the corresponding instruction channel is to be sent is not full; in response to determining that the corresponding instruction cache unit is not full, the instruction allocation control state machine enters the candidate allocation state; and generates the valid candidate allocation identifier.

[0063] At step 406, the computing device 100 caches the current computing core instruction packet in the selected current instruction channel to the computing core instruction cache unit corresponding to the computing core to which the current computing core instruction packet is to be sent.

[0064] Regarding the selected current instruction channel, it includes, for example, a state where a signal indicating that a candidate allocation is valid (e.g., pipe_vld) is pulled high, which indicates, for example, that the instruction channel corresponding to the current instruction allocation control state machine is determined to be a candidate instruction channel.

[0065] At step 408, in response to determining that the current core instruction packet in the current instruction channel has been cached, the computing device 100 selects the next instruction channel so that the next core instruction packet in the selected next instruction channel is cached in the core instruction cache unit corresponding to the core to which the next core instruction packet is to be sent.

[0066] like Figure 3 As shown, in response to determining that the current core instruction packet in the current instruction channel has been cached, the instruction channel selection module 306 identifies whether multiple instruction allocation control state machines 304 have generated a candidate allocation valid flag, and randomly selects an instruction channel from the instruction channels corresponding to the instruction allocation control state machine that has generated a candidate allocation valid flag, so as to receive the core command in the selected instruction channel and store the command in the corresponding core instruction cache unit.

[0067] Through the above-mentioned technical means, the present invention can continuously select instruction channels and cache the core instructions within the instruction channels.

[0068] In step 410, the computing device 100 determines the current computing core instruction cache unit as a candidate state in response to determining that the current computing core instruction cache unit is in a non-empty state and the corresponding computing core is in an idle state.

[0069] Regarding the compute core instruction cache being in a non-empty state, this indicates, for example, that the compute core instruction cache contains cached compute core instructions. Regarding the compute core being in an idle state, this indicates, for example, that the compute core is not currently executing any tasks and is in a standby state.

[0070] like Figure 3 As shown, the computation core cache unit selection module 310 judges the status of the current computation core instruction cache unit and the corresponding computation core (as shown by labels 320 and 322). In response to the current computation core instruction cache unit being in a non-empty state and the corresponding computation core being in an idle state, the current computation core instruction cache unit is judged to be a candidate state.

[0071] In step 412, the computing device 100 randomly selects from all candidate computing core instruction cache units via the computing core cache unit selection module 310, reads the computing core instructions from the selected computing core instruction cache unit, and sends them to the corresponding computing core.

[0072] like Figure 3 As shown, the computation core cache unit selection module 310 randomly selects from all candidate computation core instruction cache units so that the computation core instructions in the selected computation core instruction cache unit can be sent to the corresponding computation core.

[0073] In the above scheme, the computing core instruction cache unit in the computing device 100 caches the computing core instructions, which enables the allocation and issuance of computing core instructions to be carried out in parallel, thereby improving the efficiency of the issuance of computing core instructions and thus improving the actual computing performance of the artificial intelligence chip.

[0074] The following will combine Figure 3 and Figure 5 This invention describes a method 500 for allocating control instructions to a control state machine, according to an embodiment of the present invention. It should be understood that method 500 can, for example, be implemented in... Figure 1 The described computing device 100 performs the operation. Method 500 may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0075] At step 502, in response to determining that the computing core instruction of the current instruction channel is valid, the computing device 100 enters the pre-allocation state of the instruction allocation control state machine.

[0076] For example, each instruction allocation control state machine determines whether a computational core instruction exists in the corresponding instruction channel; and in response to determining that a computational core instruction exists in the corresponding instruction channel, the instruction allocation control state machine enters the pre-allocation state.

[0077] like Figure 3 As shown, multiple instruction allocation control state machines 304, in response to determining that a computational core instruction exists in the corresponding instruction channel, cause the corresponding allocation control state machine to enter the pre-allocation state. For example, as indicated by label 316, instruction allocation control state machine 7 enters the pre-allocation state in response to the presence of a computational core instruction in instruction channel 7.

[0078] At step 504, in response to determining that the instruction cache unit of the computing core to which the current instruction channel's instruction is to be sent is not full, the computing device 100 enters the candidate allocation state.

[0079] like Figure 3As shown, in response to determining that the instruction cache unit of the core to which the current instruction channel's core instruction is to be sent is not full, the instruction allocation control state machine 304 enters the candidate allocation state. For example, as indicated by label 318, in response to determining that the instruction cache unit 3 of the core to which the current instruction channel's core instruction is to be sent (e.g., core 3) is to be sent is not full, the instruction allocation control state machine 7 enters the candidate allocation state.

[0080] At step 506, in response to determining that the current instruction channel is selected, the computing device 100 enters an idle state for the instruction allocation control state machine.

[0081] For example, if the current instruction channel is selected, the core instructions within that selected instruction channel are cached in the corresponding core instruction cache unit, and the instruction allocation control state machine of that selected instruction channel enters an idle state. In some embodiments, the instruction allocation control state machine in the idle state can wait for the arrival of the next valid instruction. If a core instruction to be issued is allocated in the instruction channel corresponding to the instruction allocation control state machine, then the instruction allocation control state machine enters a pre-allocation state.

[0082] In the above scheme, the present invention configures each instruction channel with its own instruction allocation control state machine, instead of sharing the same allocation control state machine. Therefore, the allocation states between different channels do not need to be synchronized, realizing independent allocation between different instruction channels and further improving the efficiency of instruction issuance to the computing core.

[0083] The various processes and procedures described above, such as methods 200, 400, and 500, can be executed at a computing device. This computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. In some embodiments, methods 200, 400, and 500 may be implemented as computer software programs or program products tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of methods 200, 400, and 500 described above can be performed.

[0084] This invention can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention. The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0085] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. Various aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0086] These computer-readable program instructions can be provided to the central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the central processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0087] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0088] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps loaded in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0089] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors.

Claims

1. A method for allocating computation kernel instructions, characterized in that, The method includes: Select from multiple instruction channels associated with the computing core instructions to be assigned; The core instructions in the selected instruction channel are cached in the core instruction cache unit corresponding to the core to which the core instruction is to be executed. Select a non-empty instruction cache unit of the computing core, read the computing core instructions from the selected instruction cache unit, and send them to the corresponding computing core, so that instruction channel selection and computing core instruction sending can be carried out in parallel.

2. The method according to claim 1, characterized in that, Each of the multiple instruction channels is configured with its own instruction allocation control state machine, and the computation core instructions are used for multiple computation cores of the video encoding and decoding unit of the artificial intelligence chip.

3. The method according to claim 2, characterized in that, The method includes: In response to the determination that the computational core instruction of the current instruction channel is valid, the instruction allocation control state machine enters the pre-allocation state; In response to the determination that the instruction cache unit of the core to which the current instruction channel's instruction is to be sent is not full, the instruction allocation control state machine enters the candidate allocation state; and In response to the determination that the current instruction channel has been selected, the instruction allocation control state machine enters the idle state.

4. The method according to claim 1, characterized in that, The process of caching the core instructions from the selected instruction channel into the core instruction cache unit corresponding to the core to which the core instructions are to be executed includes: The current core instruction packet in the selected current instruction channel is cached in the core instruction cache unit corresponding to the core to which the current core instruction packet is to be sent; In response to determining that the current core instruction packet in the current instruction channel has been cached, the next instruction channel is selected so that the next core instruction packet in the selected next instruction channel is cached in the core instruction cache unit corresponding to the core to which the next core instruction packet is to be cached.

5. The method according to claim 1, characterized in that, The selection of multiple instruction channels associated with the computation core instructions to be assigned includes: In response to the determination that the instruction cache unit of the computation core to which the current instruction channel's instruction is to be sent is not full, the current instruction channel is determined as a candidate instruction channel, and the instruction allocation control state machine configured for the candidate instruction channel enters the candidate allocation state; and All candidate instruction channels are randomly selected to determine the selected instruction channel.

6. The method according to claim 1, characterized in that, Selecting a non-empty core instruction cache unit, reading the core instructions from the selected core instruction cache unit, and sending them to the corresponding core includes: In response to determining that the current compute core instruction cache unit is in a non-empty state and the corresponding compute core is in an idle state, the current compute core instruction cache unit is determined to be a candidate state; The computation core cache unit selection module randomly selects from all candidate computation core instruction cache units; and The core instructions are read from the selected core instruction cache unit and sent to the corresponding core.

7. The method according to claim 3, characterized in that, In response to determining that the computational core instruction for the current instruction channel is valid, the instruction allocation control state machine enters the pre-allocation state, including: Each instruction allocation control state machine determines whether a computational core instruction exists in the corresponding instruction channel; and In response to the determination that a computational core instruction exists in the corresponding instruction channel, the instruction allocation control state machine enters the pre-allocation state.

8. The method according to claim 5, characterized in that, The instruction allocation control state machine configured for the candidate instruction channel enters the candidate allocation state as follows: Each instruction allocation control state machine checks whether the instruction cache unit of the corresponding instruction core to which the instruction in the instruction channel is to be sent is not full. In response to determining that the corresponding computational core instruction cache unit is not full, the instruction allocation control state machine enters the candidate allocation state; and Generate a valid identifier for the candidate assignment.

9. The method according to claim 8, characterized in that, The process of caching the core instructions from the selected instruction channel into the core instruction cache unit corresponding to the core to which the core instructions are to be executed includes: The instruction channel selection module randomly selects the instruction channel corresponding to the instruction allocation control state machine in the candidate allocation state based on the valid candidate allocation identifier; and caches the core instructions in the selected instruction channel into the core instruction cache unit corresponding to the core to which the core instructions are to be executed.

10. An artificial intelligence chip, characterized in that, include: Multiple instruction channels are configured to allocate computing core instructions; The instruction channel selection module is configured to select from multiple instruction channels where the computing core instructions reside; The computation core instruction cache unit, corresponding to the computation core, is configured to cache the computation core instructions in the selected instruction channel into the computation core instruction cache unit corresponding to the computation core to which the computation core instructions are to be sent; as well as The compute core cache unit selection module is configured to select a compute core instruction cache unit that is not empty, and read the compute core instructions in the selected compute core instruction cache unit and send them to the corresponding compute core.

11. The artificial intelligence chip according to claim 10, characterized in that, include: Multiple instruction allocation control state machines are associated with multiple instruction channels. The instruction allocation control state machines are configured to: enter a pre-allocation state in response to determining that the computation core instruction of the current instruction channel is valid; enter a candidate allocation state in response to determining that the computation core instruction cache unit corresponding to the computation core to which the computation core instruction of the current instruction channel is to be sent is not full; and enter an idle state in response to determining that the current instruction channel is selected.

12. A computing device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method according to any one of claims 1-9.

14. A computer program product, characterized in that, Includes a computer program, which, when executed by a machine, performs the method according to any one of claims 1-9.

Images