Instruction generation method and apparatus, and electronic device

Abstract

A method and device for generating instructions and an electronic device are disclosed, and relate to the technical field of neural networks. The method comprises determining a plurality of instruction sequences corresponding to a plurality of processing units included in a neural network accelerator; based on the instruction types in the plurality of instruction sequences corresponding to the plurality of processing units, determining, in a case where the instruction types of the i-th instruction in the plurality of instruction sequences are not completely identical, a position of the i-th instruction as a to-be-processed instruction position; based on the to-be-processed instruction position, inserting an instruction of a same type as a target instruction type into a to-be-inserted instruction sequence to obtain updated plurality of instruction sequences; wherein the to-be-inserted instruction sequence is at least one instruction sequence in the plurality of instruction sequences, in which the instruction type at the to-be-processed instruction position is different from the target instruction type; and based on the updated plurality of instruction sequences, generating a composite instruction sequence of the neural network accelerator to implement instruction processing in a multi-PE mode.

Description

Technical Field

[0001] This disclosure relates to the field of neural network technology, and in particular to an instruction generation method, apparatus and electronic device. Background Technology

[0002] In chip hardware circuit design, a corresponding controller is typically assigned to each processing element (PE) within the chip. If there are multiple PEs on the chip, a separate controller is designed for each PE. Therefore, when a chip includes multiple PEs, the circuit design between the multiple PEs and multiple controllers is quite complex.

[0003] To simplify circuit design, a multi-PE mode circuit design approach has been proposed. Multi-PE mode refers to multiple physical circuits (PEs) simultaneously connected to a single controller, which then controls these PEs to execute the same type of control instructions. In practical applications of multi-PE mode, situations may arise where the instructions to be executed are not evenly distributed across the PEs. For example, if the instructions include two load instructions, and there are three PEs, some PEs may be assigned load instructions while others are not. In this case, the controller will be unable to control multiple PEs to execute the same type of control instructions simultaneously, resulting in data processing failure. Summary of the Invention

[0004] Existing neural network accelerators cannot support the processing of some instruction data in multi-PE mode.

[0005] To address the aforementioned technical problems, this disclosure is proposed. Embodiments of this disclosure provide an instruction generation method, apparatus, and electronic device that, when generating a synthetic instruction sequence executable by a neural network accelerator, fully consider the instruction types of instructions in multiple instruction sequences to be executed, thereby enabling the neural network accelerator to support the processing of all instruction data in multi-PE mode.

[0006] According to one aspect of this disclosure, a method for generating instructions for a neural network accelerator is provided, comprising: first, determining the sequence of instructions to be executed corresponding to each of the plurality of processing units included in the neural network accelerator; second, based on the instruction types in the plurality of instruction sequences to be executed corresponding to the plurality of processing units, determining the position of the i-th instruction as the position of the instruction to be processed when the instruction types of the i-th instruction in the plurality of instruction sequences to be executed are not completely the same; next, based on the position of the instruction to be processed, inserting an instruction of the same type as the target instruction into the instruction sequence to be inserted, thereby obtaining an updated plurality of instruction sequences to be executed; wherein the instruction sequence to be inserted is at least one instruction sequence in the plurality of instruction sequences to be executed where the instruction type of the instruction at the position of the instruction to be processed is different from the target instruction type; and finally, generating a synthetic instruction sequence for the neural network accelerator based on the updated plurality of instruction sequences to be executed.

[0007] Based on this scheme, a synthetic instruction sequence for the neural network accelerator is generated by considering the instruction types of multiple instruction sequences to be executed corresponding to multiple processing units. That is, when generating executable instructions for the neural network accelerator, the instruction types of the multiple instruction sequences to be executed corresponding to multiple processing units are fully considered, thus ensuring that the generated synthetic instruction sequence conforms to the data processing rules of the multi-PE mode. This transforms instructions that do not conform to the data processing rules of the multi-PE mode into instructions that do conform, thereby achieving instruction data processing.

[0008] According to one aspect of this disclosure, an instruction generation apparatus for a neural network accelerator is provided, comprising: a first determining module, configured to determine a sequence of instructions to be executed corresponding to each of a plurality of processing units included in the neural network accelerator; a second determining module, configured to, based on the instruction types in the plurality of instruction sequences to be executed corresponding to the plurality of processing units obtained by the first determining module, determine the position of the i-th instruction as the position of the instruction to be processed if the instruction types of the i-th instruction in the plurality of instruction sequences to be executed are not completely identical; an insertion module, configured to, based on the position of the instruction to be processed determined by the second determining module, insert an instruction of the same type as a target instruction into the instruction sequence to be inserted, thereby obtaining an updated plurality of instruction sequences to be executed; wherein the instruction sequence to be inserted is at least one instruction sequence in the plurality of instruction sequences to be executed where the instruction type of the instruction position to be processed is different from the target instruction type; and a generation module, configured to, based on the updated plurality of instruction sequences to be executed obtained by the insertion module, generate a synthetic instruction sequence for the neural network accelerator.

[0009] According to one aspect of this disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing any of the methods described above.

[0010] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement any of the methods described above. Attached Figure Description

[0011] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1 This is a multi-PE hardware circuit design diagram provided by an exemplary embodiment of the prior art.

[0013] Figure 2 This is a hardware circuit design diagram corresponding to the multi-PE mode provided in an exemplary embodiment of this disclosure.

[0014] Figure 3 This disclosure provides an exemplary embodiment of multiple instruction sequences corresponding to multiple PEs.

[0015] Figure 4 This is an exemplary structural diagram of the system to which this disclosure applies.

[0016] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of the present disclosure.

[0017] Figure 6 This is a schematic diagram of the structure of another electronic device provided in an exemplary embodiment of the present disclosure.

[0018] Figure 7 This is one of the flowcharts illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0019] Figure 8 This is a second schematic flowchart of an instruction generation method for a neural network accelerator provided in an exemplary embodiment of this disclosure.

[0020] Figure 9 This is the third flowchart illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0021] Figure 10 This is the fourth flowchart illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0022] Figure 11This is the fifth flowchart illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0023] Figure 12 This is a sixth flowchart illustrating an exemplary embodiment of the instruction generation method for a neural network accelerator provided in this disclosure.

[0024] Figure 13 This is one of the schematic diagrams illustrating an instruction generation method using a neural network accelerator to process multiple instruction sequences, provided in an exemplary embodiment of this disclosure.

[0025] Figure 14 This is a second schematic diagram of an instruction generation method using a neural network accelerator provided in an exemplary embodiment of this disclosure, which processes multiple instruction sequences.

[0026] Figure 15 This is a structural diagram of an instruction generation apparatus provided in an exemplary embodiment of the present disclosure.

[0027] Figure 16 This is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. Detailed Implementation

[0028] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0029] Typically, when a chip contains multiple physical processors (PEs), the hardware circuit design involves creating a separate control for each PE and connecting each PE to its corresponding control. Upon receiving a task, each control first parses the instruction to be executed and then controls the PE connected to that control to execute that instruction.

[0030] Figure 1An exemplary hardware circuit design diagram of a chip is shown. This hardware circuit includes three Controls and three PEs: Control0, Control1, and Control2; and PE0, PE1, and PE2. Control0 and PE0 are connected, Control1 and PE1 are connected, and Control2 and PE2 are connected. When Control0 receives a task, it first parses the corresponding instruction to be executed, then controls PE0 to execute the instruction. After PE0 executes the instruction, it outputs the data calculation result. The data processing methods of Control1, PE1, Control2, and PE2 are similar to those of Control0 and PE0, and will not be described further here.

[0031] Based on the above hardware circuit design, when the chip contains multiple PEs, each PE and its corresponding control must be individually wired, which is quite complex. To simplify the hardware circuit design, a multi-PE mode design is proposed. The hardware circuit design in multi-PE mode is as follows: one control corresponds to multiple PEs, that is, one control is connected to multiple PEs. This not only reduces the number of control units but also reduces the number of connections between multiple control units and PEs. The data processing rule in multi-PE mode is: one control simultaneously controls multiple PEs to execute the same type of instruction to output the data processing result.

[0032] Figure 2 An exemplary chip hardware circuit design diagram in a multi-PE mode is shown. This circuit design includes one Control and three PEs. The Control is Control0, and the three PEs are PE0, PE1, and PE2. PE0, PE1, and PE2 are all connected to Control0. After receiving a task, Control0 first parses the corresponding instruction to be executed, and then controls PE0, PE1, and PE2 to simultaneously execute instructions of the same type. For example, if Control0 parses the instruction to be executed as a load instruction, Control0 can issue a multi-PE load instruction to PE0 through PE3 to control PE0, PE1, and PE2 to execute the load instruction. This multi-PE load instruction can be: LOAD addr_pe0 = 1234, addr_pe1 = 2234, addr_pe2 = 3234.

[0033] As discussed above, multi-PE mode requires multiple PEs to simultaneously execute instructions of the same type. When the instructions cannot be evenly distributed across the PEs, or when each PE is pre-configured to handle different tasks, the sequence of instructions to be executed by each PE will inevitably differ. This can lead to a situation where a single control cannot manage multiple PEs to execute the same type of instructions, thus preventing data processing. For example, if the instructions include five load instructions, but the PEs only have three, then some PEs might be assigned two load instructions, while others might be assigned only one. This results in a single control being unable to manage multiple PEs to execute the same type of instructions, thus hindering data processing.

[0034] Figure 3 An example is shown in multi-PE mode, illustrating multiple instruction sequences corresponding to multiple PEs. These multiple instruction sequences are instruction sequence 0 for PE0, instruction sequence 1 for PE1, and instruction sequence 2 for PE2. Combined with... Figure 3 It can be seen that there are six instructions in instruction sequence 0 of PE0, seven instructions in instruction sequence 1 of PE1, and four instructions in instruction sequence 2 of PE2.

[0035] In PE0, instruction sequence 0, from the first instruction (line 1) to the last instruction (line 6), consists of Load 1, Conv 2, Store 3, Load 4, Conv 5, and Store 6. In PE1, instruction sequence 1, from the first instruction (line 1) to the last instruction (line 7), consists of Load a, Conv b, Conv c, Store d, Load e, Conv f, and Store g. In PE2, instruction sequence 2, from the first instruction (line 1) to the last instruction (line 4), consists of Load①, Conv②, Conv③, and Store④.

[0036] As discussed above, the data processing rule in multi-PE mode is that one Control unit simultaneously controls multiple PE units to execute the same type of instruction. Therefore, for... Figure 3 For the multiple instruction sequences shown, the data processing process is as follows: Control processes the first instruction (i.e., the instruction in the first line) in instruction sequence 0, instruction sequence 1, and instruction sequence 2 simultaneously, then processes the second instruction (i.e., the instruction in the second line) in instruction sequence 0, instruction sequence 1, and instruction sequence 2 simultaneously, and so on, until the last instruction in instruction sequence 0, instruction sequence 1, and instruction sequence 2 is processed.

[0037] In this sequence, the first instruction of instruction sequence 0 is Load 1, the first instruction of instruction sequence 1 is Load a, and the first instruction of instruction sequence 2 is Load①. The first instructions of instruction sequences 0, 1, and 2 are of the same type, namely Load instructions. Therefore, the first instructions of instruction sequences 0, 1, and 2 conform to the data processing rules of multi-PE mode, and Control can control PE0 to PE2 to execute the same type of Load instruction through multi-PE instructions.

[0038] The second instruction in instruction sequence 0, the second instruction in instruction sequence 1, and the second instruction in instruction sequence 2 are all Conv instructions, which also conforms to the data processing rules of multi-PE mode.

[0039] The instruction types of the third instruction in instruction sequence 0, the third instruction in instruction sequence 1, and the third instruction in instruction sequence 2 include Store instructions and Conv instructions. That is, the instruction types of the third instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2 are not completely the same. Therefore, the third instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2 do not conform to the data processing rules of multi-PE mode.

[0040] The instruction types of the fourth instruction in instruction sequence 0, the fourth instruction in instruction sequence 1, and the fourth instruction in instruction sequence 2 include Load instruction and Store instruction. That is, the instruction types of the fourth instruction in instruction sequence 0, instruction sequence 1, and instruction sequence 2 are not completely the same. Therefore, the fourth instruction in instruction sequence 0, instruction sequence 1, and instruction sequence 2 does not conform to the data processing rules of multi-PE mode.

[0041] The fifth instruction in instruction sequence 0 and the fifth instruction in instruction sequence 1 include Conv and Load instructions, and instruction sequence 2 has only four instructions and no fifth instruction. Therefore, the fifth instruction in instruction sequences 0, 1, and 2 does not conform to the data processing rules of multi-PE mode.

[0042] The sixth instruction in instruction sequence 0 and the sixth instruction in instruction sequence 1 include Store and Conv instructions, and instruction sequence 2 has only four instructions and no sixth instruction. Therefore, the fifth instruction in instruction sequences 0, 1, and 2 does not conform to the data processing rules of the multi-PE mode.

[0043] The seventh instruction in instruction sequence 1 is a Store instruction. Neither instruction sequence 0 nor instruction sequence 2 has a seventh instruction. Therefore, the seventh instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2 do not conform to the data processing rules of the multi-PE mode.

[0044] In summary, the instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2, starting from the third instruction, do not conform to the data processing rules of multi-PE mode. Therefore, in multi-PE mode, there are instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2 that cannot be processed.

[0045] Therefore, to address the issue of some instructions being unprocessable in multi-PE mode, this disclosure provides an instruction generation method for neural network accelerators. This method first determines the positions of instructions to be processed in each instruction sequence where the instruction types of the i-th instruction are not entirely identical. Since the instruction types corresponding to the positions of instructions to be processed are not entirely identical, this indicates that the positions of instructions to be processed include at least two instruction types. Because instructions from multiple PEs are processed simultaneously in multi-PE mode, the instruction types of the instructions from multiple PEs must be consistent. Therefore, it is necessary to select one instruction type from at least two instruction types as the instruction type of the instruction at the position of the instruction to be processed. The instruction sequence containing the instruction corresponding to the unselected instruction type is taken as the instruction sequence to be inserted. The instruction at the position of the instruction to be processed in the instruction sequence to be inserted is shifted down one position, and the vacated position is filled with the instruction corresponding to the selected instruction type. The instruction corresponding to the selected instruction type is an instruction in the instruction sequence to be inserted, and inserting the instruction corresponding to the selected instruction type will not destroy existing data. In this way, it can be ensured that when multiple instruction sequences corresponding to multiple PEs are processed simultaneously, the instruction types are consistent, thereby realizing data computation in multi-PE mode.

[0046] Figure 4 An embodiment of this disclosure provides a system for performing neural network operations. The system includes a compilation-side device 41 and a runtime-side device 42. The compilation-side device is used to generate a synthetic instruction sequence for the neural network, and the runtime-side device 42 is used to perform neural network operations by running the synthetic instruction sequence provided by the compilation-side device 41. These operations include, but are not limited to, convolution operations, pooling operations, activation function operations, etc.

[0047] In some embodiments, the compiler-side device 41 may be implemented by an electronic device 20. This electronic device 20 may include, but is not limited to, a personal computer, a server computer, a multiprocessor system, a microprocessor-based system, etc.

[0048] In some embodiments, the operation-side device 42 may be implemented by an electronic device 60. This electronic device 60 may include, but is not limited to, a neural network accelerator or a neural network chip.

[0049] Figure 5 This is a schematic diagram of the structure of an electronic device 50 provided in an embodiment of the present disclosure, as shown below. Figure 5 As shown, the electronic device 50 includes at least one processor 51 and a memory 52.

[0050] The processor 51 is used to execute the instruction generation method for a neural network accelerator in the embodiments of this disclosure. The processor 51 can be a high-performance central processing unit (CPU), graphics processing unit (GPU), or other processor that supports the compilation of neural network instruction sequences, or other processing units with compilation processing capability and / or instruction execution capability.

[0051] The memory 52 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random-access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 51 may execute the program instructions to implement the instruction generation method for neural network accelerators described below.

[0052] In some embodiments, such as Figure 5 As shown, the electronic device 50 may also include an input device 53 and an output device 54. The input device 53 may include, for example, a keyboard, mouse, etc., for compilers to input parameters of the neural network, such as instruction sequences of multiple PEs. The output device 54 may include, for example, a display, speaker, printer, and communication networks and their connected remote output devices, etc., for compilers to view the final compilation results or intermediate data during the compilation process.

[0053] It should be noted that the specific structure of the electronic device 50 is not limited in the embodiments disclosed herein, and the electronic device 50 may include components such as... Figure 5 The electronic device 50 may include more or fewer components. For example, it may also include components such as buses and input / output interfaces.

[0054] Figure 6 This is a schematic diagram of the structure of an electronic device 60 provided in an embodiment of the present disclosure, as shown below. Figure 6 As shown, the electronic device 60 may include a control unit 61 and a computing unit 62. The control unit 61 is used to run instructions to control the computing unit 62 to perform the following method of pooling operation on the pooling layer in the neural network model.

[0055] The control unit 61 may include one or more processors or other processing units with neural network computing capabilities and / or instruction execution capabilities. The processor may include, but is not limited to, GPUs, brain processing units (BPUs), tensor processing units (TPUs), and other processors that support neural network-related computations. In some embodiments, the control unit 61 is used to control the computing unit 62 to perform corresponding operations according to instructions from the neural network accelerator (e.g., convolution operation instructions, pooling operation instructions, and activation function operation instructions).

[0056] The computing unit 62 includes an arithmetic circuit that can be configured to perform neural network operations, including but not limited to any one or more of operations such as convolution, pooling, and activation function operations.

[0057] In some embodiments, the electronic device 60 may further include a buffer memory 63 and an off-chip memory 64. The buffer memory 63 may include one or more independent cache memories or a processing unit with data caching capabilities, and may access the off-chip memory 64 under the control of the control unit 61. In some embodiments, the buffer memory 61 may be a fast, small-capacity static access memory, such as static random access memory (SRAM). Exemplarily, the buffer memory 63 may be configured to receive at least a portion of the input feature data and / or the data of the convolution kernel from the off-chip memory 64 and cache it for use by the computing unit 62, and may also be configured to receive and cache the output feature data output by the computing unit 62, and output the output feature data to the off-chip memory 64.

[0058] The off-chip memory 64 may include one or more independent memories or processing units with data storage capabilities, and may be accessed by the buffer memory 63 under the control of the control unit 61. In some embodiments, the off-chip memory 64 is a slow, large-capacity dynamic access memory, such as dynamic random access memory (DRAM). In some embodiments, the off-chip memory 64 is double data rate (DDR).

[0059] It should be noted that the specific structure of the electronic device 60 is not limited in the embodiments disclosed herein, and the electronic device 60 may include components such as... Figure 6 The number of components shown may be more or fewer. For example, electronic device 60 may also include components such as buses.

[0060] The methods described in the following embodiments of this disclosure can be applied to many fields that require neural network computation, such as image processing and intelligent driving. For example, image data can be processed in the field of image processing by performing the methods provided in the embodiments of this disclosure.

[0061] Figure 7 This disclosure provides an instruction generation method for a neural network accelerator, which can be applied to a compiler-side device 71, such as... Figure 7 As shown, the method includes steps 701-704.

[0062] Step 701: Determine the sequence of instructions to be executed for each of the multiple processing units included in the neural network accelerator.

[0063] In some embodiments, the neural network accelerator can be any accelerator capable of executing neural network algorithms, such as a brain processing unit (BPU), a neural network processing unit (NPU), etc. This disclosure does not limit the type of neural network accelerator; the following embodiments use a BPU as an example for illustrative purposes.

[0064] For example, the aforementioned processing unit can be a Processing Equipment (PE), and multiple processing units included in the neural network accelerator can share a single controller. That is, the multiple PEs can be designed in a multi-PE mode in the hardware circuit. When the compiler inputs a task, or when the neural network accelerator triggers a task internally, the neural network accelerator assigns a task to each of the multiple PEs and generates a corresponding sequence of instructions to be executed based on the assigned task. The number of instructions included in the sequence of instructions to be executed for each of the multiple PEs may be the same or different, and this disclosure does not limit this.

[0065] For example, see Figure 2 The neural network accelerator contains multiple processing units, including PE0, PE1, and PE2. The sequence of instructions to be executed corresponding to PE0 can be... Figure 3 The instruction sequences to be executed corresponding to instruction sequences 0 and PE1 shown can be... Figure 3 The instruction sequences to be executed corresponding to instruction sequences 1 and PE2 shown can be... Figure 3 The instruction sequence shown is 2.

[0066] Step 702: Based on the instruction types in the multiple instruction sequences to be executed corresponding to multiple processing units, if it is determined that the instruction types of the i-th instruction in the multiple instruction sequences to be executed are not completely the same, the position of the i-th instruction is determined as the position of the instruction to be processed.

[0067] After determining the sequence of instructions to be executed for each of the multiple PEs, based on the data processing rules of the multi-PE mode, it is first determined whether the instructions in the multiple sequences of instructions to be executed meet the requirements.

[0068] The data processing rule for multi-PE mode is as follows: one Control unit simultaneously controls multiple PEs to execute instructions of the same type. That is, the instruction sequences corresponding to multiple PEs must satisfy the requirement that the instruction type of the i-th instruction in each sequence is the same. If the instruction types of the i-th instruction in each sequence are not completely identical (e.g., partially identical or completely different), each sequence needs to be processed to ensure that the instruction type of the i-th instruction in each sequence is the same.

[0069] It should be noted that i is an integer. The i-th instruction in each sequence of instructions to be executed is located on the same line of the instruction sequence. Therefore, we can traverse the instructions in the sequence of instructions to be executed corresponding to each PE by "line" to determine whether the instruction types of the instructions in each line of the sequence of instructions to be executed corresponding to multiple PEs are consistent.

[0070] If the instruction type is the same for each instruction in the sequence of instructions to be executed corresponding to each PE, then the multiple instruction sequences can be processed directly according to the data processing rules of the multi-PE mode to output the data operation results.

[0071] If, within the sequence of instructions to be executed corresponding to each PE, there exists a line of instructions with different instruction types, the position of that line can be determined as the position of the instruction to be processed. For example, if the instruction types of the i-th instruction in the sequence of instructions to be executed corresponding to each PE are not completely the same (i.e., the instruction types of the i-th line of instructions are not completely the same), then the position of the i-th instruction is determined as the position of the instruction to be processed. This position of the instruction to be processed can also be the position of the i-th line of instructions.

[0072] For example, see Figure 3In instruction sequence 0 corresponding to PE0, instruction sequence 1 corresponding to PE1, and instruction sequence 2 corresponding to PE2, each line of instruction sequence 0, instruction sequence 1, and instruction sequence 2 is traversed row by row. Since the instructions corresponding to the third line of instruction sequence 0, instruction sequence 1, and instruction sequence 2 are Store 3, Conv c, and Conv③; the instructions corresponding to the fourth line are Load4, Store d, and Store④; the instructions corresponding to the fifth line are Conv 5, Load e, and / ; the instructions corresponding to the sixth line are Store 6, Conv f, and / ; and the instructions corresponding to the seventh line are / , Store g, and / , where / represents that the data at that position is empty, it can be determined that the "lines" with not completely identical instruction types are the third, fourth, fifth, sixth, and seventh lines. Therefore, the positions corresponding to the third, fourth, fifth, sixth, and seventh lines of instruction sequence 0, instruction sequence 1, and instruction sequence 2 are taken as the positions of the instructions to be processed.

[0073] Step 703: Based on the position of the instruction to be processed, insert an instruction of the same type as the target instruction into the instruction sequence to be inserted, and obtain multiple updated instruction sequences to be executed.

[0074] Among them, the instruction sequence to be inserted is at least one instruction sequence among multiple instruction sequences to be executed, wherein the instruction type at the instruction position to be processed is different from the target instruction type.

[0075] In order to utilize the data processing rules of the multi-PE mode to process instructions in multiple instruction sequences, when it is determined that there are lines in multiple instruction sequences with different instruction types, an instruction of the same type as the target instruction can be inserted at the position of the instruction to be processed in the instruction sequence to be inserted (i.e., the line with different instruction types). This ensures that the instruction type at the position of the instruction to be processed in the updated multiple instruction sequences to be executed is the target instruction type.

[0076] Step 704: Generate a synthetic instruction sequence for the neural network accelerator based on the updated sequence of multiple instructions to be executed.

[0077] For example, the synthesized instructions can be multiple PE instructions. Since the instructions in the same line of the updated multiple sequences of instructions to be executed are of the same type, a multiple PE instruction sequence for the neural network accelerator can be generated based on the instructions in each line of the updated multiple sequences of instructions to be executed.

[0078] In this embodiment, the compiler-side device fully considers the instruction types of multiple instruction sequences corresponding to multiple processing units when generating executable instructions for the neural network accelerator. Therefore, the generated synthetic instruction sequence conforms to the data processing rules of the multi-PE mode. That is, it can transform instructions that do not conform to the data processing rules of the multi-PE mode into instructions that conform to the data processing rules of the multi-PE mode, thereby realizing the processing of instruction data.

[0079] In some embodiments, such as Figure 8 As shown above, in the above Figure 7 Based on the illustrated embodiment, step 703 may include steps 801-803.

[0080] Step 801: Determine the target instruction type based on the location of the instruction to be processed.

[0081] Specifically, the target instruction type is determined based on the instruction in the instruction location to be processed.

[0082] Combination Figure 3 In instruction sequence 0 corresponding to PE0, instruction sequence 1 corresponding to PE1, and instruction sequence 2 corresponding to PE2, the positions of the instructions to be processed are the positions corresponding to the third, fourth, fifth, sixth, and seventh lines in instruction sequence 0, instruction sequence 1, and instruction sequence 2, respectively.

[0083] For example, taking the third line of instruction sequences 0, 1, and 2 as the location of the instruction to be processed, the process of determining the target instruction type is as follows: Determine that the instructions corresponding to the third line of instruction sequences 0, 1, and 2 are Store 3, Conv c, and Conv③. Based on Store 3, Conv c, and Conv③, determine the target instruction type of the third line of instruction sequences 0, 1, and 2.

[0084] In some embodiments, such as Figure 9 As shown, step 801 above may include steps 901-902.

[0085] Step 901: Based on the position of the instruction to be processed, determine the instruction type of the first instruction located at the position of the instruction to be processed in each sequence of instructions to be executed.

[0086] For example, see Figure 3 Taking the third line of instruction sequence 0, instruction sequence 1 and instruction sequence 2 as the position of the instruction to be processed, the first instruction could be Store 3, Conv c or Conv③.

[0087] Step 902: Determine the target instruction type based on the instruction type of each first instruction.

[0088] If the first instruction is Store 3, and the instruction type of Store 3 is Store, then the target instruction type is Store. If the first instruction is Conv c, and the instruction type of Conv c is Conv, then the target instruction type is Conv. If the first instruction is Conv③, and the instruction type of Conv③ is Conv, then the target instruction type is also Conv.

[0089] When determining the target instruction type, any instruction type located at the position of the instruction to be processed in each sequence of instructions to be executed can be selected as the target instruction type. Alternatively, the instruction type with the highest number of occurrences in each sequence of instructions to be executed can be selected as the target instruction type. These two methods of determining the target instruction type are explained below.

[0090] In some embodiments, such as Figure 10 As shown, step 902 above may include step 1001.

[0091] Step 1001: Based on the instruction type of each first instruction, determine the instruction type of any first instruction as the target instruction type.

[0092] After determining the location of the instruction to be processed, which corresponds to multiple instructions, the first instruction can be any one of these instructions. After selecting the first instruction from among the multiple instructions, the instruction type of the first instruction is determined as the target instruction type.

[0093] For example, see Figure 3 Taking the third line of instruction sequence 0, instruction sequence 1, and instruction sequence 2 as an example where the instruction to be processed is located, the first instruction is selected from Store 3, Conv c, and Conv③. The first instruction can be any one of Store 3, Conv c, and Conv③. If the first instruction selected is Store 3, then the target instruction type is Store. If the first instruction selected is Conv c, then the target instruction type is Conv. If the first instruction selected is Conv③, then the target instruction type is also Conv.

[0094] In some embodiments, such as Figure 11 As shown above, in the above Figure 9 Based on the illustrated embodiment, step 902 may include steps 1101-1102.

[0095] Step 1101: Determine the number of instructions of the same instruction type based on the instruction type of each first instruction.

[0096] The location of instructions to be processed includes multiple instructions. These instructions can all be of different types, or they can be partially the same and partially different. If all instructions are of different types, the number of instructions of the same type is 1. If the instruction types of multiple instructions are partially the same and partially different, the number of instructions of the same type is counted to determine the number of instructions corresponding to each instruction type within the different instruction types.

[0097] For example, see Figure 3 Taking the third line of instruction sequence 0, instruction sequence 1, and instruction sequence 2 as an example where the instruction to be processed is located, the first instruction is Store 3, Conv c, and Conv ③. The instruction types of the first instruction include Store and Conv c. The number of instructions corresponding to the Store type is 1, and the number of instructions corresponding to the Conv type is 2.

[0098] Step 1102: Determine the instruction type with the largest number of instructions of the same instruction type as the target instruction type.

[0099] After determining the number of instructions of the same type, the instruction type with the most instructions is identified as the target instruction type. Using the instruction type with the most instructions as the target instruction type reduces the number of instructions to be inserted, and consequently reduces the number of insertions. Fewer insertions mean fewer errors. Therefore, this approach not only improves data processing efficiency but also reduces the error rate.

[0100] For example, see Figure 3 Taking the third line of instruction sequence 0, instruction sequence 1, and instruction sequence 2 as an example where the instruction to be processed is located, since there is 1 instruction of type Store and 2 instructions of type Conv c in the first instruction, Conv is selected as the target instruction type.

[0101] Step 802: Among multiple sequences of instructions to be executed, the sequence of instructions whose instruction type at the position of the instruction to be processed is different from the target instruction type is determined as the sequence of instructions to be inserted.

[0102] As can be seen from step 801, the target instruction type is determined based on the instructions at the instruction positions to be processed. Since the instruction types at the instruction positions to be processed in each sequence of instructions to be executed are not entirely the same, after determining the target instruction type, the instructions at the instruction positions to be processed can be divided into instructions with the same target instruction type and instructions with different target instruction types. The instruction sequence to which the instructions with different target instruction types belong is considered the instruction sequence to be inserted.

[0103] For example, see Figure 3Taking the third line of instruction sequence 0, instruction sequence 1, and instruction sequence 2 as an example, the target instruction type could be Store or Conv.

[0104] If the target instruction type is Store, the instruction with the same type as Store is Store3 in instruction sequence 0, and the instructions with different types are Conv c in instruction sequence 1 and Conv③ in instruction sequence 2. Therefore, the instruction sequences to be inserted are instruction sequence 1 and instruction sequence 2.

[0105] If the target instruction type is Conv, the instructions with the same type as Conv are Conv c in instruction sequence 1 and Conv③ in instruction sequence 2. The instruction with a different type than Conv is Store 3 in instruction sequence 0. Therefore, the instruction sequence to be inserted is instruction sequence 0.

[0106] Step 803: Insert an instruction of the same type as the target instruction into the instruction sequence to be inserted, and obtain multiple updated instruction sequences to be executed.

[0107] For example, see Figure 3 Taking the third row of instruction sequences 0, 1, and 2 as examples where the instruction to be processed is located, if the target instruction type is Store, then instructions of the same type as Store are inserted into instruction sequences 1 and 2, resulting in updated instruction sequences 1 and 2; the instruction type in the third row of the updated instruction sequences 0, 1, and 2 is all Store. If the target instruction type is Conv, then instructions of the same type as Conv are inserted into instruction sequence 0, resulting in updated instruction sequence 0; the instruction type in the third row of the updated instruction sequences 0, 1, and 2 is all Conv.

[0108] In some embodiments, such as Figure 12 As shown above, in the above Figure 8 Based on the embodiment shown, step 803 above includes steps 1201-1202.

[0109] Step 1201: Based on the target instruction type, determine the instruction with the same type as the target instruction in the instruction sequence to be inserted.

[0110] Combination Figure 10 and Figure 11It can be seen that there are two ways to determine the target instruction type. After determining the target instruction type, the corresponding instruction is identified from the instruction sequence to be inserted. The found instruction must satisfy at least two conditions: first, the instruction type is the target instruction type; second, the insertion of the instruction does not disrupt the data operation results of the instruction sequence to be inserted, ensuring that the data operation results after the insertion are the same as before the insertion.

[0111] Step 1202: Insert an instruction of the same type as the target instruction at the position of the instruction to be processed in the instruction sequence to be inserted.

[0112] In use Figure 11 Once the target instruction type is determined as shown, it can be referenced as an example. Figure 13 The first processing scheme shown inserts an instruction of the same type as the target instruction at the position of the instruction to be processed in the instruction sequence to be inserted.

[0113] Figure 13 Example (a) illustrates instruction sequence 0 of PE0, instruction sequence 1 of PE1, and instruction sequence 2 of PE2. The instruction types of the first line of instruction data in instruction sequences 0, 1, and 2 are all consistent, and can be processed based on the multi-PE mode data processing rules. The instruction types of the second line of instruction data in instruction sequences 0, 1, and 2 are also consistent, and can be processed based on the multi-PE mode data processing rules. The instruction types of the third line of instruction data in instruction sequences 0, 1, and 2 include Store and Conv. To improve data processing efficiency, the following can be used... Figure 11 As shown, the target instruction type in the third line of instruction sequence 0, instruction sequence 1, and instruction sequence 2 is determined to be Conv.

[0114] Based on the target instruction type Conv, the instruction sequence to be inserted is determined to be instruction sequence 0. Within instruction sequence 0, instructions consistent with the target instruction type Conv include Conv 2 and Conv 5. Since Conv 2 is closer to the third line of instruction sequence 0 than Conv 5, inserting Conv 2 into the third line of instruction sequence 0 is equivalent to repeatedly calculating Conv 2, which will not corrupt the data operation result of instruction sequence 0. Therefore, the instruction consistent with the target instruction type Conv is determined to be Conv 2 in instruction sequence 0.

[0115] Next, insert Conv 2 into the third line of instruction sequence 0, and... Figure 13 In instruction sequence 0 indicated by identifier 11 in (b), the instruction data of the third line and subsequent lines are shifted down by one bit, resulting in the following: Figure 13Instruction sequence 0 is shown in (c). For the third line and the positions of the shifted instruction data after the third line in instruction sequence 0, see [reference needed]. Figure 13 The box corresponding to identifier 11 in (c) of the document.

[0116] After inserting the target instruction in the third line, it is necessary to continue traversing the fourth line of instructions in each instruction sequence. If the instruction types of the fourth line instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2 are the same, then there is no need to update the fourth line instructions; multiple PE instructions can be generated directly. If the instruction types of the fourth line instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2 are not completely the same, then it is necessary to follow the above procedure. Figures 7 to 12 The instruction generation method shown in any embodiment updates the fourth line of instructions and generates the corresponding multi-PE instructions. This process is repeated until the last line of instructions is processed, resulting in three updated sequences of instructions to be executed, where each line of instructions in the three updated sequences has the same instruction type.

[0117] After performing the above processing on the instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2, lines 4 to 7, we can obtain the following: Figure 13 The updated instruction sequences (d) are shown in the diagram: Updated instruction sequence 0, Updated instruction sequence 1, and Updated instruction sequence 2. In Updated instruction sequence 0, the instructions from line 1 to line 7 are Load 1, Conv 2, Conv 2, Store 3, Load 4, Conv 5, and Store 6, respectively. In Updated instruction sequence 1, the instructions from line 1 to line 7 are Load a, Conv b, Conv c, Store d, Load e, Conv f, and Store g, respectively. In Updated instruction sequence 2, the instructions from line 1 to line 7 are Load ①, Conv ②, Conv ③, Load ④, Load ①, Conv ③, and Load ④, respectively. Clearly, the instruction types in the same line are the same in Updated instruction sequences 0, 1, and 2.

[0118] After obtaining the updated sequences of instructions to be executed, the instructions in each line of the updated sequences are merged to generate the synthetic instruction sequence of the neural network accelerator.

[0119] Specifically, the instruction data in the updated instruction sequence 0, instruction sequence 1, and instruction sequence 2 are merged to obtain the composite instruction sequence, i.e. Figure 13The multi-PE instruction is shown in (e). The multi-PE instruction includes 7 sub-PE instructions, namely Load 1, a, ①; Conv 2, b, ②; Conv 2, c, ③; Store 3, d, ④; Load 4, e, ①; Conv 5, f, ③; and Store 6, g, ④.

[0120] In use Figure 10 Once the target instruction type is determined as shown, it can be referenced as an example. Figure 14 The second processing scheme shown inserts an instruction of the same type as the target instruction at the position of the instruction to be processed in the instruction sequence to be inserted.

[0121] based on Figure 13 Example (a) shows instruction sequence 0 of PE0, instruction sequence 1 of PE1, and instruction sequence 2 of PE2. First, the instruction data in the third line of instruction sequences 0, 1, and 2 is adjusted. The instruction types in the third line of instruction data in instruction sequences 0, 1, and 2 include Store and Conv. Using... Figure 10 The method shown allows you to determine the target instruction type as Store or Conv in the third line of instruction sequence 0, instruction sequence 1, and instruction sequence 2.

[0122] For example, the second processing scheme will be explained using the example that the target instruction type in the third line of instruction sequence 0, instruction sequence 1 and instruction sequence 2 is Store.

[0123] Based on the target instruction type Store, the instruction sequences to be inserted are identified as instruction sequence 1 and instruction sequence 2. Therefore, instructions matching the target instruction type Store need to be identified in both instruction sequence 1 and instruction sequence 2. In instruction sequence 1, the instructions matching the target instruction type Store are Store d and Store g. Since Store d is closer to the third line of instruction sequence 1 than Store g, Store d is identified as the instruction matching the target instruction type Store in instruction sequence 1. In instruction sequence 2, the only instruction matching the target instruction type Store is Store ④. Therefore, Store ④ is identified as the instruction matching the target instruction type Store in instruction sequence 2.

[0124] Next, insert Store d into the third line of instruction sequence 1, insert Store④ into the third line of instruction sequence 2, and then... Figure 14 In instruction sequence 1, as indicated by identifier 12 in (a), the original third line and subsequent instruction data, as well as the original third line and subsequent instruction data in instruction sequence 2, are all shifted down by one position, resulting in the following: Figure 14 Instruction sequence 1 and instruction sequence 2 are shown in (b) above. The instruction data in instruction sequence 1, from the third line onwards, and the instruction data in instruction sequence 2, from the third line onwards, are shown below. The shifted positions are shown in [reference needed]. Figure 14 The box corresponding to identifier 12 in (b) of the diagram.

[0125] After inserting the target instruction in the third line, it is necessary to continue traversing the fourth line of instructions in each instruction sequence. If the instruction types of the fourth line instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2 are the same, then there is no need to update the fourth line instructions; multiple PE instructions can be generated directly. If the instruction types of the fourth line instructions in instruction sequence 0, instruction sequence 1, and instruction sequence 2 are not completely the same, then it is necessary to follow the above procedure. Figures 7 to 12 The instruction generation method shown in any embodiment updates the fourth line of instructions and generates the corresponding multi-PE instructions. This process is repeated until the last line of instructions is processed, resulting in three updated sequences of instructions to be executed, where each line of instructions in the three updated sequences has the same instruction type.

[0126] In some embodiments, when calculating instructions using the data processing rules of the multi-PE mode, the data operation results of the instruction at the instruction to be processed position in the instruction sequence to be inserted can be stored in an inactive address, while the data operation results of other instructions in the original instruction sequence to be inserted (excluding the instruction at the instruction to be processed position, i.e., instructions in the instruction sequence to be executed when no instruction is inserted) can be stored in an active address. The inactive address is used to store invalid data, and the active address is used to store valid data.

[0127] For example, consider a sequence of instructions to be inserted that includes instructions 1 and 2, with instruction 3 to be inserted between instructions 1 and 2. If the results of the operations on instructions 1, 2, and 3 are all stored at active addresses (e.g., active addresses 1, 2, and 3 respectively), then when instruction 2 is executed, the result of the operation on instruction 3 will be read from active address 3 and processed according to the instruction type of instruction 2. This differs from the original sequence of instructions to be inserted, where the result of the operation on instruction 1 is read from active address 1 and processed according to the instruction type of instruction 2. Therefore, the result of the operation after inserting instruction 3 will differ from the result before inserting instruction 3.

[0128] If the result of instruction 3 is stored at an inactive address (e.g., inactive address 1), and the results of instructions 1 and 2 are stored at active addresses 1 and 2 respectively, then when instruction 2 is executed, since the result of instruction 3 stored at inactive address 1 is invalid, the result of instruction 1 will be read from active address 1 and processed according to the instruction type of instruction 2. This is exactly the same as when the original sequence of instructions to be inserted reads the result of instruction 1 from active address 1 and processes it according to the instruction type of instruction 2. Therefore, the result of the data operation after inserting instruction 3 is exactly the same as the result of the data operation without inserting instruction 3.

[0129] In other words, this application stores the data operation result of the inserted instruction in an inactive address so that the data operation result of the inserted instruction will not participate in the operation of each instruction in the original instruction sequence to be inserted. Therefore, it can ensure that the data operation result after the inserted instruction is the same as the data operation result after the instruction is not inserted.

[0130] Figure 15 An instruction generation apparatus provided in this disclosure embodiment, such as Figure 15 As shown, the instruction generation device 1500 includes a first determination module 1501, a second determination module 1502, an insertion module 1503, and a generation module 1504.

[0131] The first determining module 1501 is used to determine the sequence of instructions to be executed for each of the multiple processing units included in the neural network accelerator.

[0132] The second determining module 1502 is used to determine the position of the i-th instruction as the position of the instruction to be processed when the instruction types of the i-th instruction in the multiple sequences of instructions to be executed corresponding to the multiple processing units obtained by the first determining module are not completely the same.

[0133] The insertion module 1503 is used to insert an instruction of the same type as the target instruction into the instruction sequence to be inserted based on the instruction position to be processed determined by the second determining module, so as to obtain an updated sequence of multiple instruction sequences to be executed; wherein, the instruction sequence to be inserted is at least one instruction sequence among the multiple instruction sequences to be executed, in which the instruction type at the instruction position to be processed is different from the target instruction type.

[0134] The generation module 1504 is used to generate a synthetic instruction sequence for the neural network accelerator based on the updated sequence of multiple instructions to be executed obtained from the insertion module.

[0135] In some embodiments, the insertion module 1503 includes a first determining unit, a second determining unit, and a third determining unit. The first determining unit is configured to determine a target instruction type based on the position of the instruction to be processed. The second determining unit is configured to determine, among multiple sequences of instructions to be executed, the sequence of instructions whose instruction type at the position of the instruction to be processed is different from the target instruction type determined by the second determining unit as the sequence of instructions to be inserted. The third determining unit is configured to insert an instruction of the same type as the target instruction into the sequence of instructions to be inserted determined by the second determining unit, thereby obtaining an updated sequence of multiple sequences of instructions to be executed.

[0136] In some embodiments, the first determining unit includes a first determining subunit and a second determining subunit. The first determining subunit is used to determine the instruction type of a first instruction located at the instruction position in each sequence of instructions to be executed, based on the instruction position to be processed. The second determining subunit is used to determine the target instruction type based on the instruction type of each first instruction.

[0137] In some embodiments, the second determining subunit is specifically configured to determine the instruction type of any first instruction as the target instruction type based on the instruction type of each first instruction.

[0138] In some embodiments, the second determining subunit is specifically configured to determine the number of instructions of the same instruction type based on the instruction type of each first instruction; and to determine the instruction type with the largest number of instructions of the same instruction type as the target instruction type.

[0139] In some embodiments, the third determining unit is specifically configured to determine, according to the target instruction type, an instruction of the same type as the target instruction in the instruction sequence to be inserted; and to insert an instruction of the same type as the target instruction at the position of the instruction to be processed in the instruction sequence to be inserted, thereby obtaining an updated sequence of multiple instructions to be executed.

[0140] Figure 16 A structural diagram of an electronic device provided in this disclosure embodiment, such as... Figure 16 As shown, the electronic device 1600 includes one or more processors 1601 and memory 1602.

[0141] The processor 1601 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1600 to perform desired functions.

[0142] The memory 1602 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1601 may execute the program instructions to implement the instruction generation methods of the various embodiments of this disclosure described above and / or other desired functions.

[0143] In one example, the electronic device 1600 may also include an input device 1603 and an output device 1604, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0144] Of course, for the sake of simplicity, Figure 16 Only some of the components of the electronic device 1600 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1600 may include any other suitable components depending on the specific application.

[0145] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the instruction generation methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0146] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0147] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the instruction generation methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0148] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0149] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0150] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0151] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0152] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0153] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for generating instructions for a neural network accelerator, comprising: The sequence of instructions to be executed for each of the multiple processing units included in the neural network accelerator is determined; wherein the multiple processing units share a single controller; Based on the instruction types in the multiple sequences of instructions to be executed corresponding to the multiple processing units, if it is determined that the instruction types of the i-th instruction in the multiple sequences of instructions to be executed are not completely the same, the position of the i-th instruction is determined as the position of the instruction to be processed. Based on the position of the instruction to be processed, an instruction of the same type as the target instruction is inserted into the instruction sequence to be inserted, resulting in multiple updated instruction sequences to be executed; wherein, the instruction sequence to be inserted is at least one instruction sequence among the multiple instruction sequences to be executed, where the instruction type at the position of the instruction to be processed is different from the target instruction type; Based on the updated sequence of multiple instructions to be executed, a synthetic instruction sequence for the neural network accelerator is generated; wherein, the instructions in the same row of the updated sequence of multiple instructions to be executed have the same type.

2. The method according to claim 1, wherein, The step of inserting an instruction of the same type as the target instruction into the instruction sequence to be inserted based on the position of the instruction to be processed, resulting in multiple updated sequences of instructions to be executed, includes: Based on the location of the instruction to be processed, the type of the target instruction is determined; Among the plurality of instruction sequences to be executed, the instruction sequence whose instruction type at the instruction position to be processed is different from the target instruction type is determined as the instruction sequence to be inserted; Insert an instruction of the same type as the target instruction into the instruction sequence to be inserted, resulting in multiple updated instruction sequences to be executed.

3. The method according to claim 2, wherein, Determining the target instruction type based on the location of the instruction to be processed includes: Based on the position of the instruction to be processed, determine the instruction type of the first instruction located at the position of the instruction to be processed in each sequence of instructions to be executed; The target instruction type is determined based on the instruction type of each of the first instructions.

4. The method according to claim 3, wherein, Determining the target instruction type based on the instruction type of each of the first instructions includes: Based on the instruction type of each of the first instructions, the instruction type of any one of the first instructions is determined as the target instruction type.

5. The method according to claim 3, wherein, Determining the target instruction type based on the instruction type of each of the first instructions includes: Based on the instruction type of each of the first instructions, determine the number of instructions of the same instruction type; The instruction type with the largest number of instructions of the same instruction type is determined as the target instruction type.

6. The method according to any one of claims 2-5, wherein, The step of inserting an instruction of the same type as the target instruction into the instruction sequence to be inserted, resulting in an updated sequence of multiple instructions to be executed, includes: Based on the target instruction type, determine the instruction with the same type as the target instruction in the instruction sequence to be inserted; Insert the instruction of the same type as the target instruction at the position of the instruction to be processed in the instruction sequence to be inserted, and obtain the updated sequence of multiple instructions to be executed.

7. An apparatus for instruction generation in a neural network accelerator, comprising: The first determining module is used to determine the sequence of instructions to be executed corresponding to each of the multiple processing units included in the neural network accelerator; wherein the multiple processing units share a single controller; The second determining module is used to determine the position of the i-th instruction as the position of the instruction to be processed when the instruction types of the i-th instruction in the multiple sequences of instructions to be executed corresponding to the multiple processing units obtained by the first determining module are not completely the same; An insertion module is configured to insert an instruction of the same type as the target instruction into the instruction sequence to be inserted based on the position of the instruction to be processed determined by the second determining module, thereby obtaining an updated sequence of multiple instruction sequences to be executed; wherein, the instruction sequence to be inserted is at least one instruction sequence among the multiple instruction sequences to be executed, wherein the instruction type at the position of the instruction to be processed is different from the target instruction type; A generation module is used to generate a synthetic instruction sequence for the neural network accelerator based on the updated multiple instruction sequences to be executed obtained by the insertion module; wherein the instructions in the same row of the updated multiple instruction sequences to be executed have the same type.

8. The apparatus according to claim 7, wherein, The insertion module includes: The first determining unit is used to determine the target instruction type based on the position of the instruction to be processed; The second determining unit is configured to determine, among the plurality of instruction sequences to be executed, the instruction sequence whose instruction type at the position of the instruction to be processed is different from the target instruction type determined by the second determining unit as the instruction sequence to be inserted; The third determining unit is used to insert an instruction of the same type as the target instruction into the instruction sequence to be inserted determined by the second determining unit, so as to obtain an updated sequence of multiple instructions to be executed.

9. A computer-readable storage medium storing a computer program for performing the instruction generation method for a neural network accelerator as described in any one of claims 1-6.

10. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the instruction generation method for a neural network accelerator as described in any one of claims 1-6.

Images