Instruction processing apparatus, instruction processing method, and chip system

Abstract

The embodiment of the invention provides an instruction processing device, an instruction processing method and a chip system. The instruction processing device comprises a data control module configured to receive and process a loop initial instruction to obtain a loop descriptor; the loop processing module is configured to construct a loop according to the loop descriptor and generate loop level information and control information for realizing loop control; and the instruction control module is configured to generate an executable instruction of the neural network processor according to the loop level information and the control information, the executable instruction is provided for the neural network processor to execute, and the instruction control module corresponds to the loop processing module. According to the instruction processing device, efficient, high-robustness and high-expansibility instruction control and generation can be carried out, the instruction sending efficiency is improved, the load of the coprocessor can be relieved, the performance of the coprocessor can be released for other functions, and the maximum utilization of system resources is achieved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to an instruction processing device, an instruction processing method, and a chip system. Background Art

[0002] A neural-network processing unit (NPU) is a type of microprocessor or computing system dedicated to hardware acceleration of artificial intelligence (especially artificial neural networks, machine vision, machine learning, etc.), sometimes also referred to as an artificial intelligence accelerator (AI Accelerator).

[0003] In the NPU system architecture that adopts the instruction-driven mechanism, especially in the complex row-stationary (RS) NPU system architecture, a set of instruction sets is equipped to cope with the processing requirements of the internal loop structure of the RS NPU. The instruction set can cover operations such as data handling, loop end control, and padding. Summary of the invention

[0004] At least one embodiment of the present disclosure provides an instruction processing device, which includes: a data control module, configured to receive and process a loop initialization instruction to obtain a loop descriptor; a loop processing module, configured to construct a loop according to the loop descriptor, and generate loop level information and control information for implementing loop control; an instruction control module, configured to generate a neural network processor executable instruction according to the loop level information and control information, so as to provide it to the neural network processor for execution, wherein the instruction control module corresponds to the loop processing module.

[0005] For example, in the instruction processing device provided in at least one embodiment of the present disclosure, it also includes: an instruction distribution module, which is configured to classify the executable instructions of the neural network processor and send them to the corresponding subsequent module of the instruction processing device.

[0006] For example, in the instruction processing device provided by at least one embodiment of the present disclosure, the data control module includes an interface management unit and a loop descriptor cache unit, the interface management unit is configured to parse and disassemble the received loop initialization instruction to obtain the loop descriptor, and store the loop descriptor into the loop descriptor cache unit; the loop descriptor cache unit is configured to store the loop descriptor.

[0007] For example, in the instruction processing device provided by at least one embodiment of the present disclosure, the loop processing module includes: a loop descriptor analysis unit, configured to convert the loop descriptor into a loop structure and operations corresponding to each layer of the loop, and generate the loop level information, wherein the loop level information includes the loop level and loop value of each level of the loop; a first loop control unit, configured to perform loop control operations, wherein the loop control operations include loop valid level jump control, empty level jump mechanism and / or corresponding level control information management; a second loop control unit, configured to indicate the current loop level according to the loop control operation of the first loop control unit, and control the increase or decrease of the loop value corresponding to the current loop level.

[0008] For example, in the instruction processing device provided by at least one embodiment of the present disclosure, the instruction control module includes: a loop parameter register file, configured to store the loop level information; an instruction parameter register file, configured to store the control information of the loop level currently indicated by the second loop control unit; an instruction generation unit, configured to receive the control information of the corresponding loop level from the instruction parameter register file, and generate an executable instruction for the neural network processor according to the control information of the corresponding loop level.

[0009] For example, in the instruction processing device provided by at least one embodiment of the present disclosure, the instruction control module also includes: a filling control unit, which is configured to generate a filling instruction for the current loop level based on the dimension field and the dimension filling field in the loop initial instruction and the loop dimension field corresponding to the current loop level.

[0010] For example, in the instruction processing device provided by at least one embodiment of the present disclosure, the instruction control module also includes: a pipeline control unit, configured to pipeline control a plurality of the instruction generation units; a configuration status unit, configured to store configuration information of the instruction processing device, wherein the configuration information includes information corresponding to the loop initial instruction set configuration encoding; a segmented instruction generation unit, configured to receive control information of the corresponding loop level from the instruction parameter register file, and generate segmented instructions according to the control information of the corresponding loop level.

[0011] For example, in the instruction processing device provided by at least one embodiment of the present disclosure, the instruction distribution module includes: a distribution channel, configured to classify and schedule executable instructions of the neural network processor; a descriptor decoding control unit, configured to decode the loop level information; a configuration instruction control unit, configured to generate a configuration instruction based on the decoded loop level information received from the descriptor decoding control unit, and send it to the subsequent module for register configuration.

[0012] For example, in the instruction processing device provided in at least one embodiment of the present disclosure, the configuration instruction control unit is further configured to generate a polling instruction after the register configuration is completed, and send the polling instruction to the subsequent module to monitor the status information of the subsequent module.

[0013] For example, in the instruction processing device provided in at least one embodiment of the present disclosure, the distribution channel includes an instruction control unit, a first instruction queue and a second instruction queue; the instruction control unit is configured to receive and parse the neural network processor executable instructions, and distribute the neural network processor executable instructions to the first instruction queue or the second instruction queue according to the instruction type of the neural network processor executable instructions, and the neural network processor executable instructions include standard instructions and custom instructions; the first instruction queue is configured to receive and send the standard instructions to the subsequent module; the second instruction queue is configured to receive and send the custom instructions to the subsequent module.

[0014] For example, in the instruction processing device provided in at least one embodiment of the present disclosure, it also includes: a status recording unit, configured to record the status information of each module in the instruction processing device and the subsequent module, and to interact with the outside of the instruction processing device through an event interface; an event control unit, configured to implement event synchronization and control according to the status information.

[0015] For example, in the instruction processing device provided in at least one embodiment of the present disclosure, it also includes: a subsequent configuration control unit configured to configure the subsequent module; and a cross switch unit configured to support the subsequent configuration control unit to interact with the subsequent module.

[0016] For example, in the instruction processing device provided in at least one embodiment of the present disclosure, it also includes: a matrix descriptor analysis unit, configured to parse matrix dimension information according to the obtained matrix multiplication instruction to obtain loop information; a matrix control unit, configured to generate the loop initialization instruction according to the loop information.

[0017] At least one embodiment of the present disclosure provides an instruction processing method, which includes: receiving and processing a loop initialization instruction to obtain a loop descriptor; constructing a loop according to the loop descriptor, generating loop level information and control information, and implementing loop control; generating a neural network processor executable instruction according to the loop level information and control information, for providing to the neural network processor for execution.

[0018] For example, in the instruction processing method provided in at least one embodiment of the present disclosure, it also includes: classifying the executable instructions of the neural network processor and sending them to the corresponding subsequent modules.

[0019] For example, in the instruction processing method provided by at least one embodiment of the present disclosure, the loop initial instruction includes: a dimension field, which represents the dimension information of the calculation data; and at least one loop initial instruction segment, wherein each of the at least one loop initial instruction segment corresponds to a loop level and includes: a loop level field, which represents the loop level where the current loop is located; a loop value field, which represents the number of times the current loop initial instruction segment needs to loop; and a loop dimension field, which represents the dimension in which the current loop initial instruction segment is calculated.

[0020] For example, in the instruction processing method provided by at least one embodiment of the present disclosure, the loop initialization instruction also includes: a dimension filling field, which represents the dimension extension information of the calculation data.

[0021] For example, in the instruction processing method provided by at least one embodiment of the present disclosure, each of the at least one loop initial instruction segment further includes: a space control field indicating whether space allocation is performed in the current loop and the direction of the space allocation.

[0022] For example, in the instruction processing method provided by at least one embodiment of the present disclosure, each of the at least one loop initial instruction segment further includes: a custom field, which is set at the starting end or the ending end of the loop initial instruction segment and is configured as a custom instruction.

[0023] For example, in the instruction processing method provided by at least one embodiment of the present disclosure, each of the at least one loop initial instruction segment further includes: an invalid data encoding field for indicating the location of invalid data in the calculation data of the current loop.

[0024] For example, in the instruction processing method provided by at least one embodiment of the present disclosure, the loop initialization instruction also includes: a loop initialization instruction set configuration code, which is configured to indicate a valid loop level and set an instruction set parameter bit width.

[0025] At least one embodiment of the present disclosure provides a chip system, which includes: an instruction processing device provided according to at least one embodiment of the present disclosure; and a neural network processor configured to execute executable instructions of the neural network processor.

[0026] For example, in the chip system provided by at least one embodiment of the present disclosure, it also includes: a processor configured to send the loop initialization instruction to the instruction processing device. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0028] Figure 1 A schematic block diagram of an instruction processing device provided for at least one embodiment of the present disclosure;

[0029] Figure 2 A schematic diagram of an example of a loop structure provided for at least one embodiment of the present disclosure;

[0030] Figure 3 A schematic block diagram of an instruction processing device provided for at least one embodiment of the present disclosure;

[0031] Figure 4 A flowchart of an instruction processing method provided for at least one embodiment of the present disclosure;

[0032] Figure 5 A flowchart of an instruction processing method provided for at least one embodiment of the present disclosure;

[0033] Figure 6 A schematic diagram of a loop initialization instruction provided for at least one embodiment of the present disclosure;

[0034] Fig. 7A A schematic block diagram of a chip system provided for at least one embodiment of the present disclosure; and

[0035] Figure 7B A schematic block diagram of a chip system provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0037] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0038] The present disclosure is described below through several specific embodiments. In order to keep the following description of the embodiments of the present disclosure clear and concise, the detailed description of known functions and known components (elements) may be omitted. When any part (element) of the embodiments of the present disclosure appears in more than one figure, the part (element) is represented by the same or similar reference numeral in each figure.

[0039] Traditional system-on-chip (SoC) architectures usually only include a central processing unit (CPU) and an NPU. In this architecture, the CPU acts as the main controller, responsible for managing the entire system-on-chip, and the NPU is used to perform neural network computing tasks (such as convolution calculations, etc.). However, in this architecture, the CPU is heavily loaded, resulting in a decrease in the performance of the entire system. This is because in addition to conventional management and control tasks, the CPU also needs to coordinate a large amount of data transmission and complex neural network computing tasks. To improve this problem, a common approach is to introduce a coprocessor (usually one or more small CPUs) into the SoC architecture, which is specifically used to control the NPU, thereby reducing the burden on the CPU as the main controller.

[0040] Row Stationary (RS) data flow is a data layout and computation strategy designed for NPUs, especially when processing convolution operations of convolutional neural networks (CNNs). The processing unit array in RS NPUs (such as Eyeriss architecture NPUs) implements row stationary data flow by reusing rows of filter weights horizontally between processing units and rows of input feature map feature values ​​diagonally between processing units.

[0041] In the NPU system architecture that adopts the instruction-driven mechanism, especially in the complex RS NPU system architecture, a set of instruction sets is equipped to cope with the processing requirements of the internal loop structure of the RS NPU. The instruction set can cover operations such as data handling, loop end control, and filling.

[0042] The inventors of the present disclosure note that although the complexity and flexibility of the above loop structure design are high, the overly complex loop logic will lead to a sharp increase in the number of instructions, and a large number of complex instruction streams will be generated during actual operation. On the other hand, after the coprocessor is introduced into the traditional SoC architecture, although the performance of the system is improved to a certain extent, the function and load capacity of the coprocessor are also limited. When faced with complex and large neural network models, a large number of instructions will significantly increase the load of the coprocessor, so that its instruction sending ability becomes the performance bottleneck of the entire system. At the same time, this also increases the complexity and difficulty of software-level implementation.

[0043] At least one embodiment of the present disclosure provides an instruction processing device, which includes a data control module, a loop processing module and an instruction control module. The data control module is configured to receive and process loop initialization instructions to obtain a loop descriptor; the loop processing module is configured to construct a loop according to the loop descriptor, generate loop level information and control information for implementing loop control; the instruction control module is configured to generate a neural network processor executable instruction according to the loop level information and control information, for providing to the neural network processor for execution, and the instruction control module corresponds to the loop processing module.

[0044] The instruction processing device provided by at least one embodiment of the present disclosure can perform efficient, highly robust, and highly scalable instruction control and generation, improve instruction sending efficiency, reduce coprocessor load, release coprocessor performance for other functions, and maximize system resource utilization. Furthermore, the instruction processing device can achieve efficient pipeline processing and reduce hardware overhead.

[0045] Figure 1 A schematic block diagram of an instruction processing device provided by at least one embodiment of the present disclosure. Figure 1 As shown, the instruction processing device 100 includes a data control module 101 , a loop processing module 102 and an instruction control module 103 .

[0046] like Figure 1 As shown, the data control module 101 is configured to receive and process a loop initialization instruction to obtain a loop descriptor.

[0047] For example, the loop initial instruction may be an instruction issued by a coprocessor. For example, the loop initial instruction may be a convolutional neural network instruction (CNN Instruction), which usually involves complex operations such as convolution, pooling, and activation functions.

[0048] For example, the loop descriptor can be obtained by parsing and disassembling the loop initial instruction. The loop descriptor may include information describing the loop structure, such as the loop level, the number of loops, and the loop dimension, which is not limited in this embodiment of the present disclosure.

[0049] like Figure 1 As shown, the loop processing module 102 is configured to construct a loop according to the loop descriptor, and generate loop level information and control information for implementing loop control.

[0050] For example, the loop processing module 102 may be one or more and may be set as needed. For example, a first loop processing module and a second loop processing module may be set, the first loop processing module is used to process input feature map data (ifmap), and the second loop processing module is used to process weight data (weight).

[0051] like Figure 1 As shown, the instruction control module 103 is configured to generate executable instructions for the neural network processor according to the loop level information and the control information, so as to provide the instructions to the neural network processor for execution. The instruction control module 103 corresponds to the loop processing module 102.

[0052] For example, the instruction control module 103 is configured to generate a neural network processor executable instruction based on the loop level information and control information generated by the above-mentioned loop processing module 102, so as to provide it to the neural network processor for execution. For example, the instruction control module 103 can be one or more, and can be set as needed, and it is only necessary to ensure that the instruction control module 103 corresponds to the loop processing module 102 one by one. For example, in the above example, a first instruction control module and a second instruction control module can be set, and the first instruction control module corresponds to the first loop processing module, and generates a neural network processor executable instruction for processing input feature map data; the second instruction control module corresponds to the second loop processing module, and generates a neural network processor executable instruction for processing weight data.

[0053] For example, a neural network processor includes read direct memory access (RDMA), global buffer (GLB), processing unit cluster (PE Cluster, PEC) and other post-stage modules. RDMA allows data to be directly transferred between system memory and internal storage of neural network memory (such as static random access memory SRAM or dynamic random access memory DRAM). The global buffer is used to temporarily store data such as weights, activation values, and intermediate calculation results of the neural network model. It plays the role of a data exchange center, enabling different processing unit clusters to share data, reduce dependence on external storage, and improve data reuse and computing efficiency. The processing unit cluster is a module composed of multiple processing units (PEs), each of which is responsible for performing basic computing operations of the neural network, such as matrix multiplication, vector addition, and activation functions. The clustered processing unit design can achieve highly parallel processing and accelerate the calculation of the neural network layer. By increasing the storage space that each processing unit can directly use and connecting them through an efficient interconnection network, the processing unit cluster can act as a relatively independent computing unit, while reducing data movement and improving the overall computing throughput.

[0054] In the instruction processing device provided in at least one embodiment of the present disclosure, it also includes an instruction distribution module, which is configured to classify the executable instructions of the neural network processor and send them to the corresponding subsequent modules of the instruction processing device.

[0055] For example, the executable instructions of a neural network processor can be classified according to the functional characteristics and application purposes in a neural network processor or similar computing architecture. For example, the executable instructions of a neural network processor can be divided into standard instructions and custom instructions. Standard instructions usually include basic operations that are predefined, widely applicable and well-defined in hardware design, such as data handling instructions, loop end control instructions, filling instructions, etc.; custom instructions are designed to meet special needs in specific models or computing layers, and are defined at the software level to solve the problem that standard instructions may not be sufficient to efficiently execute certain highly optimized or novel algorithm operations in some complex neural network structures. Different types of executable instructions for neural network processors are sent to different post-level modules for processing through preset interfaces.

[0056] In the instruction processing device provided by at least one embodiment of the present disclosure, the data control module 101 includes an interface management unit and a loop descriptor cache unit.

[0057] In some examples, the interface management unit is configured to parse and disassemble the received loop initialization instruction to obtain a loop descriptor, and store the loop descriptor in a loop descriptor cache unit. The interface management unit is a bridge between the coprocessor and the instruction processing device, organizes the parsed instruction information into a format suitable for the internal processing logic, and stores it in the loop descriptor cache unit according to a predetermined rule, ensuring that subsequent processing stages can efficiently access and execute.

[0058] In some examples, the loop descriptor cache unit is configured to store loop descriptors. In some examples, the loop descriptor cache unit can be a Ping-Pong Cache. For example, the Ping-Pong Cache includes at least two caches, each of which can read and write data independently. In practical applications, the number of caches can also be increased as needed. For example, when a cache is being written to a loop descriptor by an interface management unit, another cache can be read by other modules, and when the read and write operations are completed, the two roles are exchanged. By switching between the two caches, the idle waiting time between data reading and data writing can be avoided, the concurrent processing capability of the instruction processing device is enhanced, and the instruction processing device can efficiently process continuous instruction streams and data streams, ensuring the efficient execution of neural network calculations and the full utilization of resources.

[0059] In the instruction processing device provided by at least one embodiment of the present disclosure, the loop processing module 102 includes a loop descriptor analysis unit, a first loop control unit and a second loop control unit.

[0060] In some examples, the loop descriptor analysis unit is configured to convert the loop descriptor into a loop structure and operations corresponding to each loop layer, and generate loop level information, the loop level information including loop levels and loop values ​​of each loop layer. For example, the loop value indicates the number of times the instructions of the current loop level need to be executed in a loop.

[0061] Figure 2 A schematic diagram of an example of a loop structure provided in at least one embodiment of the present disclosure, using a multi-level loop structure to implement parallel computing. Figure 2 As shown in the figure, G3, N3 and other labels represent various loop levels, and the maximum utilization of resources is achieved through various time allocations and space allocations. Figure 2 As shown in the figure, SRAM time allocation refers to the data access at the SRAM level involved in the G3~C3 loop level, which is usually used to load input feature maps and filter weights into memory. GLB time allocation refers to the data access at the global buffer level involved in the G2~M2 loop level. SPAD is the internal storage of the processing unit, which is used to store active data, that is, the F0~M0 loop level involves the inside of the processing unit.

[0062] The following are some examples of how the loop descriptor analysis unit converts the loop descriptor into operations corresponding to each layer of loop. Figure 2 The F0 level in includes fill instructions, loop end control instructions, and the data step length (for data handling instructions) when jumping to the next valid loop level. The data step length is calculated by the loop descriptor analysis unit based on the loop dimension field and dimension field in the loop initial instruction. For example, Figure 2 The E3 level in includes the data step size when it jumps to the next valid loop level. For example, Figure 2 The G2 level in includes the loop end control instruction and the data step when jumping to the next valid loop level. For example, Figure 2 The F3 level in includes custom instructions.

[0063] For example, the loop levels allocated in the PEC-Y space are converted into processing unit cluster identifiers (PECIDs) in the instructions, for example, the C1 level and the R2 level correspond to C1×R2 different processing unit cluster identifiers, respectively. One processing unit cluster identifier corresponds to one column of the processing unit cluster array.

[0064] For example, the levels allocated in the PEC-X space are linked to different instruction processing channels. Figure 2 In the example of , the number of instruction processing channels is equal to G1×N1×M1×E2. One instruction processing channel corresponds to a row of the processing unit cluster array. The instruction processing channel and the processing unit cluster identifier can be used to locate which processing unit cluster executes the instruction of the corresponding loop level.

[0065] In some examples, the first loop control unit is configured to perform loop control operations, wherein the loop control operations include loop effective level jump control, empty level jump mechanism and / or corresponding level control information management. For example, loop effective level jump control refers to directly jumping to the loop level that has actual operation requirements at the current moment to start execution, thereby avoiding unnecessary computational overhead. For example, the empty level jump mechanism means that there may be an empty level in the loop structure that does not perform any operation. The first loop control unit can identify and skip the empty level and directly enter the next valid loop level. For example, the control information can be the number of loops of the corresponding level, the dimension information of the corresponding level, the instruction information of the corresponding level, the address and length of the data fetched, etc., which is not limited in the embodiments of the present disclosure. The first loop control unit can ensure the accurate execution of the loop operation and provide a basis for dynamic adjustment and optimization.

[0066] In some examples, the second loop control unit is configured to indicate the current loop level according to the loop control operation of the first loop control unit, and control the increase or decrease of the loop value corresponding to the current loop level. For example, the second loop control unit indicates the loop level to be executed currently through a pointer, and directly controls the increase or decrease of the loop counter of the level, and the value of the loop counter is the loop value. For example, when the current loop level operation indicated by the second loop control unit completes a round of iteration, the loop value corresponding to the current loop level will be automatically reduced by one. The second loop control unit can accurately control each iteration of the loop through the pointer and loop value management mechanism, thereby optimizing the allocation and use of computing resources.

[0067] In the instruction processing device provided by at least one embodiment of the present disclosure, the instruction control module 103 includes a loop parameter register file, an instruction parameter register file, and an instruction generation unit.

[0068] In some examples, the loop parameter register file is configured to store loop level information, i.e., loop levels and loop values ​​of each level of loop. The second loop control unit above generates a pointer control corresponding to the loop parameter register file and controls the increase or decrease of the loop value. The loop parameter register file can also be configured to determine the bit width of a single level, for example, set to 16 bits or 32 bits, thereby achieving the expansion of the level.

[0069] In some examples, the instruction parameter register file is configured to store control information of the loop level currently indicated by the second loop control unit. For example, the control information is stored in the instruction parameter register file by the loop descriptor analysis unit. The instruction parameter register file can be implemented by increasing the actual memory space, or by overwriting the loop descriptor cache unit, so as to save memory space.

[0070] For example, multiple loop parameter register files and multiple loop parameter register files can be set according to the needs of space allocation. Each loop parameter register file-loop parameter register file pair is associated with an instruction processing channel, ensuring that each instruction processing channel can independently manage the corresponding loop level information and control information. Multiple instruction processing channels correspond to multiple rows of the processing unit cluster array in the subsequent neural network processor. In order to achieve spatial allocation of loops, different loop levels can be assigned to multiple instruction processing channels, and the subsequently generated instructions will be assigned to multiple rows of the neural network processor processing unit cluster array for execution, so that different parts of the loop can be executed in parallel, reducing waiting time and improving execution efficiency.

[0071] In some examples, the instruction generation unit is configured to receive control information of the corresponding loop level from the instruction parameter register file and generate an executable instruction for the neural network processor according to the control information of the corresponding loop level. For example, one or more instruction generation units may be provided, and the number of the instruction generation units may be adjusted according to the design goal, which is not limited in the embodiments of the present disclosure.

[0072] The purpose of setting up multiple instruction generation units is to solve the potential problem of mismatching front-end and back-end processing speeds through a parallel processing mechanism. When instruction generation becomes a bottleneck, multiple instruction generation units can work simultaneously to ensure that the throughput of the entire system is not limited by the working speed of the instruction generation unit. Multiple instruction generation units can realize pipeline operations, that is, each instruction unit processes instruction generation tasks of different loop levels at different time points. For example, suppose that it takes 10 clock cycles to generate instructions, while it only takes 1 clock cycle to generate loop level information and control information. In the absence of a parallel mechanism, this will cause other parts of the instruction processing device to idle and wait for a long time. However, if 10 parallel instruction generation units are set up, each unit starts working at a different clock cycle. In theory, it can ensure that new instructions are generated in each clock cycle, thereby matching the generation speed of loop level information and control information, solving the problem of mismatching front-end and back-end processing speeds, and greatly improving processing efficiency.

[0073] In the instruction processing device provided by at least one embodiment of the present disclosure, the instruction control module 103 further includes a filling control unit.

[0074] In some examples, the filling control unit is configured to generate a filling instruction for the current loop level according to the dimension field and the dimension filling field in the loop initial instruction and the loop dimension field corresponding to the current loop level. For example, the dimension field indicates the dimension information of the calculation data, the dimension filling field indicates the dimension extension information of the calculation data, and the loop dimension field indicates the dimension in which the current loop initial instruction segment is calculated.

[0075] In the instruction processing device provided by at least one embodiment of the present disclosure, the instruction control module 103 further includes a pipeline control unit, a configuration state unit and a segmented instruction generation unit.

[0076] In some examples, the pipeline control unit is configured to pipeline control multiple instruction generation units. Through the scheduling of the pipeline control unit, each instruction generation unit can execute tasks in parallel or sequentially, reducing the waiting time between instructions and achieving maximum efficiency.

[0077] In some examples, the configuration state unit is configured to store configuration information of the instruction processing device, the configuration information including information corresponding to the loop initial instruction set configuration code. For example, the loop initial instruction set configuration code is configured to indicate a valid loop level and set the instruction set parameter bit width.

[0078] In some examples, the segmentation instruction generation unit is configured to receive control information of the corresponding loop level from the instruction parameter register file, and generate the segmentation instruction according to the control information of the corresponding loop level. The segmentation instruction is a custom instruction. For a scenario where the step size increases regularly, a segmentation instruction can be parsed into multiple instructions by read direct memory access (RDMA).

[0079] In the instruction processing device provided by at least one embodiment of the present disclosure, the instruction distribution module includes a distribution channel, a descriptor decoding control unit and a configuration instruction control unit.

[0080] In some examples, the distribution channel is configured to classify and schedule the neural network executable instructions. For example, multiple distribution channels can be set according to the needs of space allocation. A loop parameter register file-loop parameter register file pair and a distribution channel in the above text jointly implement an instruction processing channel, ensuring that the neural network executable instructions generated by the instruction generation unit (which instruction processing channel corresponds to can be determined based on the channel information carried in the instruction) will be distributed to multiple rows of the neural network processor processing unit cluster array for execution through the instruction processing channel, so that different parts of the loop can be executed in parallel, reducing waiting time and improving execution efficiency.

[0081] In some examples, the descriptor decoding control unit is configured to decode loop level information. For example, the descriptor decoding control unit decodes the loop level information parsed from the loop descriptor cache and sends it to the configuration instruction control unit.

[0082] In some examples, the configuration instruction control unit is configured to generate a configuration instruction based on the decoded loop level information received from the descriptor decoding control unit, and send it to the subsequent module for register configuration. The configuration instruction control unit is also configured to generate a polling instruction after the register configuration is completed, and send the polling instruction to the subsequent module to monitor the status information of the subsequent module.

[0083] For example, the configuration instruction control unit sends the configuration instruction to the subsequent module through the first configuration interface for register configuration. After the register configuration is completed, the configuration instruction control unit sends the polling instruction to the subsequent module through the first configuration interface, and the monitored status information of the subsequent module is returned to the configuration instruction control unit by the first configuration interface. For example, the first configuration interface can be based on the AXI Lite protocol, or other protocols can be selected according to actual needs, which is not limited in the embodiments of the present disclosure.

[0084] In the instruction processing device provided by at least one embodiment of the present disclosure, the distribution channel includes an instruction control unit, a first instruction queue and a second instruction queue.

[0085] In some examples, the instruction control unit is configured to receive and parse neural network executable instructions, and distribute the neural network executable instructions to a first instruction queue or a second instruction queue according to the instruction type of the neural network executable instructions, and the neural network executable instructions include standard instructions and custom instructions. Standard instructions generally include basic operations that are predefined, widely applicable, and well-defined in hardware design, such as data handling instructions, loop end control instructions, filling instructions, etc.; custom instructions are designed to meet special needs in specific models or computing layers, and are defined at the software level to solve the problem that standard instructions may not be sufficient to efficiently execute certain highly optimized or novel algorithmic operations in certain complex neural network structures. The introduction of custom instructions enables the system to be highly customized for specific applications or optimization requirements, thereby improving the robustness and scalability of the hardware design.

[0086] Different types of neural network processor executable instructions are sent to different subsequent modules for processing. In some examples, the first instruction queue is configured to receive and send standard instructions to the subsequent module, and the second instruction queue is configured to receive and send custom instructions to the subsequent module. For example, the first instruction queue and the second instruction queue interact with the subsequent module through an instruction generation interface.

[0087] At least one embodiment of the present disclosure provides an instruction processing device further comprising an event synchronization module, which is configured to coordinate event synchronization between different hardware modules. The event synchronization module comprises a state recording unit, an event control unit, a post-configuration control unit and a cross switch unit.

[0088] In some examples, the state recording unit is configured to record the state information of each module and the subsequent modules in the instruction processing device, and to perform state interaction with the outside of the instruction processing device through the event interface. For example, the state information may be the working state of the module, the task completion status, the error or abnormal condition, etc. For example, the state recording unit may perform information interaction with the processor or coprocessor through the event interface, so that the external processor or coprocessor can monitor the internal operation status of the instruction processing device and the subsequent modules in real time.

[0089] In some examples, the event control unit is configured to implement event synchronization and control based on the state information. For example, the event control unit can determine when to start an operation, pause or resume module operation, and how to respond to specific events to ensure synchronization of various modules. For example, the event control unit can implement synchronization of the instruction processing device, the subsequent module, and the processor or coprocessor and other master control units.

[0090] In some examples, the post-stage configuration control unit is configured to configure the post-stage module. For example, the post-stage configuration control unit receives configuration information through the second configuration interface and sends the parsed result to the configuration instruction control unit. For example, the post-stage configuration control unit can dynamically adjust the working parameters, function settings, etc. of the post-stage module to adapt to different task requirements or optimize system performance.

[0091] In some examples, the crossbar switch unit is configured to support the interaction between the subsequent configuration control unit and the subsequent modules. For example, the crossbar switch unit supports the selection and switching of multiple signal paths, thereby enabling connection and interaction with multiple subsequent modules, ensuring that the configuration instructions issued by the subsequent configuration control unit can reach the corresponding subsequent modules.

[0092] The instruction processing device provided by at least one embodiment of the present disclosure further includes a matrix conversion module, which is configured to receive and process a matrix multiplication instruction and generate a loop initialization instruction. For example, the matrix conversion module can receive a matrix multiplication instruction sent by an external coprocessor. The matrix conversion module includes a matrix descriptor analysis unit and a matrix control unit.

[0093] In some examples, the matrix descriptor analysis unit is configured to parse matrix dimension information according to the obtained matrix multiplication instruction to obtain loop information. For example, an example of a matrix multiplication instruction may be multiplying two matrices of sizes M×K and K×N, respectively. The matrix descriptor analysis unit parses the matrix dimension information M, K, and N from the matrix multiplication instruction, and decomposes the matrix dimension information according to a preset priority and a preset threshold to obtain loop information.

[0094] For example, an example of decomposing M, K, and N according to preset rules is as follows:

[0095] (1) K = C3 × C1 × C2 × CV × C0, the decomposition priority is CV> C1> C0> C2> C3, the preset threshold is C0≤32, C1≤4, CV≤PE_Y, where PE_Y is the number of columns of the processing unit array in each processing unit cluster of the neural network processor.

[0096] (2) N = M3 × M1 × M2 × M0, M0 ≤ 16, M2 × M0 × E3 ≤ ACC_BUFFER_SIZE, where ACC_BUFFER_SIZE is the size of the accumulation buffer of the neural network processor

[0097] (3) M = E4 × E2 × E3 × E1, the decomposition priority is E1>E2>E3>E4, and the preset thresholds are E1≤PE_X, E2≤PEC_X, E3×C2×CV×C0≤GLB_SIZE, where PE_X is the number of rows of the processing unit array in each processing unit cluster of the neural network processor, PEC_X is the number of rows of the processing unit cluster array in the neural network processor, and GLB_SIZE is the size of the global buffer of the neural network processor.

[0098] By decomposing the matrix dimension information M, K, N, the values ​​of E1-E4, C0-C3, CV, M0-M3 are the cycle information. It should be noted that the above preset threshold is determined according to the design requirements of the subsequent module and is only an example, and the embodiment of the present disclosure does not limit this.

[0099] In some examples, the matrix control unit is configured to generate a loop initialization instruction according to the loop information. For example, the loop initialization instruction generated by the matrix control unit is sent to the data control module, and the interface management unit is configured to parse and disassemble the loop initialization instruction to obtain a loop descriptor, and store the loop descriptor in the loop descriptor cache unit for subsequent instruction generation.

[0100] Figure 3 A schematic block diagram of an instruction processing device provided for at least one embodiment of the present disclosure.

[0101] like Figure 3 As shown, the input image is converted into loop initialization instructions and matrix multiplication instructions by the coprocessor, and sent by the coprocessor to the instruction processing device for processing.

[0102] like Figure 3 As shown, the instruction processing device includes two instruction control modules, which are used to generate a neural network processor executable instruction for processing input feature map data and a neural network processor executable instruction for processing weight data. It should be noted that the number of the above instruction control modules is only an example and can be determined according to actual needs.

[0103] like Figure 3 As shown, the instruction processing device includes three loop parameter register files, three instruction parameter register files and three distribution channels to meet the needs of space allocation, that is, each group of loop parameter register files, instruction parameter register files and distribution channels corresponds to one instruction processing channel, and there are three instruction processing channels in total. For details, please refer to the above description. It should be noted that the number of the above loop parameter register files, instruction parameter register files and distribution channels is only an example and can be determined according to actual needs.

[0104] Figure 3The other modules and units in the are all introduced in detail above, please refer to the description above, and will not be repeated here.

[0105] Figure 4 A flowchart of an instruction processing method provided by at least one embodiment of the present disclosure. Figure 4 As shown, the instruction processing method provided by the embodiment of the present disclosure includes the following steps S101 to S103.

[0106] Step S101: receiving and processing a loop initialization instruction to obtain a loop descriptor.

[0107] For example, the data control module provided in the above embodiment may be used to receive and process the loop initialization instruction to obtain the loop descriptor.

[0108] Step S102: construct a loop according to the loop descriptor, generate loop level information and control information, and implement loop control.

[0109] For example, the loop processing module provided in the above embodiments may be used to construct a loop according to a loop descriptor, generate loop level information and control information, and implement loop control.

[0110] Step S103: Generate a neural network processor executable instruction according to the loop level information and the control information, so as to provide it to the neural network processor for execution.

[0111] For example, the instruction control module in the above embodiment can be used to generate executable instructions for a neural network processor according to loop level information and control information, so as to provide the instructions to the neural network processor for execution.

[0112] The instruction processing method provided by at least one embodiment of the present disclosure further includes the following step S104.

[0113] Step S104: classify the neural network processor executable instructions and send them to the corresponding subsequent modules.

[0114] For example, the instruction distribution module in the above embodiment can be used to classify the executable instructions of the neural network processor and send them to the corresponding subsequent modules.

[0115] Figure 5 A flowchart of an instruction processing method provided by at least one embodiment of the present disclosure. Figure 5 As shown, the instruction processing method provided by the embodiment of the present disclosure includes the following steps S201 to S217.

[0116] Step S201: command and configuration input.

[0117] Step S202: the interface management unit parses and disassembles the received loop initialization instruction to obtain a loop descriptor, and stores it in the loop descriptor cache unit, and then executes step S205.

[0118] Step S203: The matrix conversion module receives and processes the matrix multiplication instruction, generates a loop initialization instruction, and executes step S202.

[0119] Step S204: The post-configuration control unit receives the configuration information through the second configuration interface, parses and decodes it, and executes step S214.

[0120] Step S205: The loop descriptor analysis unit converts the loop descriptor into a loop structure and operations corresponding to each layer of loop, and generates loop level information.

[0121] Step S206: Refresh the loop parameter register file and instruction parameter register file cache.

[0122] Step S207: The first loop control unit and the second loop control unit perform loop control and pointer control.

[0123] Step S208: The loop parameter register file stores the loop level information.

[0124] Step S209: the instruction parameter register file stores the control information of the loop level currently indicated by the second loop control unit.

[0125] Step S210: The filling control unit generates a filling instruction for the current loop level.

[0126] Step S211: The instruction generation unit generates executable instructions for the neural network processor.

[0127] Step S212: The instruction distribution module classifies the executable instructions of the neural network processor and sends them to the subsequent module, and executes step S216.

[0128] Step S213: The descriptor decoding control unit decodes the loop level information.

[0129] Step S214: The configuration instruction control unit generates a configuration instruction based on the decoded loop level information, and sends it to the subsequent module for register configuration.

[0130] Step S215: The event control unit implements event synchronization and control according to the status information, and executes step S212.

[0131] Step S216: The configuration instruction control unit monitors the status of the subsequent modules.

[0132] Step S217: The status recording unit records the status information of each module and interacts with the outside through the event interface.

[0133] In at least one embodiment of the present disclosure, the loop initialization instruction may include: a dimension field, indicating the dimension information of the calculation data; and at least one loop initialization instruction segment. Each of the at least one loop initialization instruction segment corresponds to a loop level and includes: a loop level field, indicating the loop level of the current loop; a loop value field, indicating the number of times the current loop initialization instruction segment needs to loop; and a loop dimension field, indicating the dimension in which the current loop initialization instruction segment is calculated.

[0134] For example, the dimension field is used to represent the dimension information of the calculation data. If the calculation data includes four dimensions, N, C, H, and W, the dimension field records the specific values ​​of each of the four dimensions. For example, N=32, C=3, H=224, and W=224, which means that there are 32 images with a size of 224×224 pixels and three channels (RGB) as input.

[0135] For example, the loop level field is used to identify the specific loop level corresponding to the loop initial instruction segment, such as level 0, level 1, etc., to ensure that the instruction sequence is correctly executed in the nested loop. For example, when the loop level field is 0x01, it indicates that the current loop is a level 1 loop.

[0136] For example, the loop value field indicates how many times the current loop initial instruction segment needs to be looped and executed. For example, when the loop value field is 0b01, it indicates that the loop needs to be executed once, and when the loop value field is 0b11, it indicates that the loop needs to be executed three times.

[0137] For example, the calculation data includes four dimensions: N, C, H, and W. When the loop dimension field is the first value (for example, 0b00), it indicates that the calculation is performed in the N dimension. When the loop dimension field is the second value (for example, 0b01), it indicates that the calculation is performed in the C dimension. When the loop dimension field is the third value (for example, 0b10), it indicates that the calculation is performed in the H dimension. When the loop dimension field is the fourth value (for example, 0b11), it indicates that the calculation is performed in the W dimension.

[0138] For example, the loop initialization instruction provided in at least one embodiment further includes a dimension filling field, which indicates the dimension expansion information of the calculation data. For example, the dimension filling field allows the data size to be expanded based on the original calculation data dimension to meet the calculation requirements. If the calculation data includes four dimensions of N, C, H, and W, the dimension filling field can specify how much data (0 or other default values) to fill in each dimension.

[0139] For example, each of the at least one loop initial instruction segment provided by at least one embodiment may further include a space control field. The space control field indicates whether the current loop performs space allocation and the direction of space allocation.

[0140] For example, in the physical space, there is a two-dimensional array composed of multiple space blocks (Packs), such as m space blocks in the X-axis direction and n space blocks in the Y-axis direction (a total of m×n space blocks), and the data required for the current cycle can be flexibly allocated and parallel processing can be performed according to different settings of the space control field. An example may be the PEC-X space and the PEC-Y space in the above embodiment.

[0141] For example, when the space control field is the first value (for example, 0b00), it indicates that no space allocation is performed; when the space control field is the second value (for example, 0b01), it indicates that the data required for the current loop is divided into m parts, which are respectively allocated to m space blocks in the X-axis direction to achieve parallel processing; when the space control field is the third value (for example, 0b10), it indicates that the data required for the current loop is divided into n parts, which are respectively allocated to n space blocks in the Y-axis direction to achieve parallel processing; when the space control field is the fourth value (for example, 0b11), it indicates that the data required for the current loop is divided into m×n parts, which are respectively allocated to m×n space blocks to achieve parallel processing.

[0142] The spatial control field determines how data is distributed in physical space to facilitate the parallel execution of cyclic tasks. It can flexibly adapt to the parallel computing needs in different scenarios, thereby greatly improving the speed of processing large amounts of data.

[0143] For example, each of the at least one loop initial instruction segment provided in at least one embodiment may further include a custom field, which is set at the start or end of the loop initial instruction segment and is configured as a custom instruction. For example, by setting a custom field, complex business logic or specific processing steps can be abstracted into a custom instruction, providing a certain degree of scalability.

[0144] For example, each of the at least one loop initial instruction segment provided by at least one embodiment may further include an invalid data encoding field, which is used to indicate the location of invalid data in the calculation data of the current loop. For example, the calculation data may have holes, and the invalid data encoding field is used to indicate where the calculation data has holes, that is, how much invalid empty data information is contained in the calculation data of the current loop, which can be used for subsequent accurate calculation of the step length of the read data.

[0145] Figure 6 A schematic diagram of a loop initialization instruction provided for at least one embodiment of the present disclosure.

[0146] For example, Figure 6As shown, the dimension field NCHW Info represents the dimension information of the calculated data, and the dimension filling fields NPadding Info, C Padding Info, H Padding Info, and W Padding Info respectively represent how much data needs to be filled in the four dimensions of N, C, H, and W. Loop0, Loop1, ..., Loopx are all loop initial instruction segments. In each loop initial instruction segment, the loop level field ID is used to identify the specific loop level corresponding to the loop initial instruction segment, the loop value field Num indicates how many times the current loop initial instruction segment needs to be executed in a loop, the loop dimension field Loop Info indicates which of the N, C, H, and W dimensions is calculated, the space control field Space indicates whether the current loop initial instruction segment needs to be allocated space and the direction of space allocation, the invalid data encoding field Invalid Num Encode is used to indicate the location of invalid data in the calculation data of the current loop, and the custom field Customize Instruction Define is set at the end of the loop initial instruction segment for custom instructions.

[0147] For example, the loop initialization instruction provided in at least one embodiment above may also include a loop initialization instruction set configuration code, and the loop initialization instruction set configuration code is configured to indicate a valid loop level. For example, the loop initialization instruction set configuration code includes a valid level field, which can be represented by a set of one-hot codes (One-Hot), and the invalid loop level corresponds to the first value (e.g., 0), and the valid loop level corresponds to the second value (e.g., 1). For example, assuming that there are 8 loop levels (0 to 7), if loop levels 0, 1, 6, and 7 are valid, and other loop levels (2 to 5) are invalid, then the one-hot code is represented as 11000011, where the 0th, 1st, 6th, and 7th bits of the binary number correspond to loop levels 0, 1, 6, and 7, respectively, and their values ​​are 1, indicating that these levels are valid, and the values ​​of other bits (2nd to 5th bits) are 0, indicating that these levels are invalid. It should be noted that the size of the valid level field can be set according to actual needs, such as 32bit, 64bit, etc., and the embodiments of the present disclosure are not limited to this.

[0148] For example, the loop initial instruction set configuration code is also configured to set the instruction set parameter bit width. For example, the loop initial instruction set configuration code includes a parameter bit width selection field. For example, when the parameter bit width selection field is a first value (for example, 0), the parameter bit width of the instruction is set to 16 bits; when the parameter bit width selection field is a second value (for example, 1), the parameter bit width of the instruction is set to 32 bits. When the parameter bit width of the instruction is set to 16 bits, a maximum of 64 layers of loops are supported. It should be noted that the parameter bit width of the instruction is set to 16 bits / 32 bits is only an example, and can also be set to other sizes according to actual needs. The embodiments of the present disclosure are not limited to this.

[0149] By configuring the initial instruction set of the loop, the attribute configuration of the instruction itself can be realized to indicate the number of loop levels.

[0150] Fig. 7A A schematic block diagram of a chip system provided for at least one embodiment of the present disclosure.

[0151] For example, Fig. 7A As shown, at least one embodiment of the present disclosure provides a chip system 700, and the chip system 700 includes an instruction processing device 701 and a neural network processor 702. For example, the instruction processing device 701 is an instruction processing device provided by at least one embodiment of the present disclosure, and can be Figure 1 Instruction processing device 100 in. For example, neural network processor 702 is configured to execute neural network processor executable instructions. For example, neural network processor 702 may be a microprocessor or computing system for artificial intelligence hardware acceleration. For example, neural network processor 702 may be implemented based on application specific integrated circuit (ASIC) or field programmable gate array (FPGA), etc., or may be integrated in system chip (SoC). For example, neural network processor 702 may be a fixed row neural network processor (RS NPU). For example, neural network processor 702 may include a processing unit (PE) array, a global cache, and a memory. The processing unit array includes multiple rows and columns of processing units, which are coupled to each other through on-chip interconnects and share a global cache, such as a network on chip (NoC). Each processing unit has a computing function, and for example, may also have its own local cache, such as a cache or register array including a multiplier accumulator and a vector (or matrix) for caching input. Each processing unit can access other processing units around it, the local cache of the processing unit itself, and the global cache. The global cache is further coupled to the memory through, for example, a bus. The instruction processing device 701 and the neural network processor 702 can be connected via a custom interface.

[0152] Figure 7BA schematic block diagram of a chip system provided for at least one embodiment of the present disclosure. For example, the chip system 700 provided in at least one embodiment of the present disclosure also includes a processor 703. For example, the processor 703 is configured to send a loop initialization instruction to the instruction processing device 701. For example, the processor may be a coprocessor, a central processing unit (CPU), or a general-purpose graphics processing unit (GPGPU), etc. For example, the instruction processing device 701 may be driven and configured by the processor 703, and the instruction processing device 701 and the processor 703 may be connected via an on-chip interconnect, such as an on-chip network (NoC). In addition, it is also possible to choose to connect via a custom interface. Although the present disclosure has been described in detail above with general instructions and specific implementation methods, it is obvious to those skilled in the art that some modifications or improvements may be made to it based on the embodiments of the present disclosure. Therefore, these modifications or improvements made without departing from the spirit of the present disclosure are all within the scope of protection claimed in the present disclosure.

[0153] There are a few points to note about this disclosure:

[0154] (1) The drawings of the embodiments of the present disclosure only relate to the structures related to the embodiments of the present disclosure, and other structures may refer to the general design.

[0155] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present disclosure, the thickness of layers or regions is enlarged or reduced, that is, these drawings are not drawn according to the actual scale.

[0156] (3) In the absence of conflict, the embodiments of the present disclosure and the features therein may be combined with each other to obtain new embodiments.

[0157] The above description is only a specific implementation of the present disclosure, but the protection scope of the present disclosure is not limited thereto. The protection scope of the present disclosure shall be based on the protection scope of the claims.

Claims

1. An instruction processing device, comprising: The data control module is configured to receive and process a loop initialization instruction to obtain a loop descriptor; A loop processing module, configured to construct a loop according to the loop descriptor, and generate loop level information and control information for implementing loop control; An instruction control module is configured to generate executable instructions for a neural network processor based on the loop level information and control information, so as to provide the instructions to the neural network processor for execution, wherein the instruction control module corresponds to the loop processing module.

2. The instruction processing device according to claim 1, further comprising: The instruction distribution module is configured to classify the executable instructions of the neural network processor and send them to the subsequent modules corresponding to the instruction processing device.

3. The instruction processing device according to claim 1, wherein: The data control module includes an interface management unit and a loop descriptor cache unit. The interface management unit is configured to parse and disassemble the received loop initialization instruction to obtain the loop descriptor, and store the loop descriptor in the loop descriptor cache unit; The loop descriptor cache unit is configured to store the loop descriptor.

4. The instruction processing device according to claim 1, wherein: The loop processing module includes: A loop descriptor analysis unit, configured to convert the loop descriptor into a loop structure and operations corresponding to each layer of loop, and generate the loop level information, wherein the loop level information includes loop levels and loop values ​​of each level of loop; A first loop control unit, configured to perform loop control operations, wherein the loop control operations include loop valid level jump control, empty level jump mechanism and / or corresponding level control information management; The second loop control unit is configured to indicate a current loop level according to the loop control operation of the first loop control unit, and control the increase or decrease of a loop value corresponding to the current loop level.

5. The instruction processing device according to claim 4, wherein: The command control module comprises: a loop parameter register file configured to store the loop level information; an instruction parameter register file, configured to store control information of a loop level currently indicated by the second loop control unit; The instruction generation unit is configured to receive control information of the corresponding loop level from the instruction parameter register file, and generate executable instructions for the neural network processor according to the control information of the corresponding loop level.

6. The instruction processing device according to claim 5, wherein: The command control module also includes: The filling control unit is configured to generate a filling instruction for the current loop level according to the dimension field and the dimension filling field in the loop initial instruction and the loop dimension field corresponding to the current loop level.

7. The instruction processing device according to claim 5, wherein: The command control module also includes: A pipeline control unit, configured to pipeline control a plurality of the instruction generation units; A configuration state unit, configured to store configuration information of the instruction processing device, wherein the configuration information includes information corresponding to the loop initial instruction set configuration code; The segment instruction generating unit is configured to receive control information of a corresponding loop level from the instruction parameter register file, and generate a segment instruction according to the control information of the corresponding loop level.

8. The instruction processing device according to claim 2, wherein: The instruction distribution module includes: A distribution channel configured to classify and schedule the executable instructions of the neural network processor; A descriptor decoding control unit configured to decode the loop level information; The configuration instruction control unit is configured to generate a configuration instruction based on the decoded loop level information received from the descriptor decoding control unit, and send the configuration instruction to the subsequent module for register configuration.

9. The instruction processing device according to claim 8, wherein: The configuration instruction control unit is further configured to generate a polling instruction after the register configuration is completed, and send the polling instruction to the subsequent module to monitor the status information of the subsequent module.

10. The instruction processing device according to claim 8, wherein: The distribution channel includes an instruction control unit, a first instruction queue and a second instruction queue; The instruction control unit is configured to receive and parse the neural network processor executable instructions, and distribute the neural network processor executable instructions to the first instruction queue or the second instruction queue according to the instruction type of the neural network processor executable instructions, wherein the neural network processor executable instructions include standard instructions and custom instructions; The first instruction queue is configured to receive and send the standard instruction to the subsequent module; The second instruction queue is configured to receive and send the custom instruction to the subsequent module.

11. The instruction processing apparatus according to claim 2, further comprising: A status recording unit, configured to record status information of each module in the instruction processing device and the subsequent module, and to perform status interaction with the outside of the instruction processing device through an event interface; The event control unit is configured to implement event synchronization and control according to the state information.

12. The instruction processing apparatus according to claim 11, further comprising: A post-stage configuration control unit, configured to configure the post-stage module; The cross switch unit is configured to support the subsequent configuration control unit to interact with the subsequent modules.

13. The instruction processing apparatus according to claim 1, further comprising: A matrix descriptor analysis unit is configured to parse matrix dimension information according to the obtained matrix multiplication instruction to obtain loop information; The matrix control unit is configured to generate the loop initialization instruction according to the loop information.

14. A method for processing an instruction, comprising: Receive and process loop initialization instructions to obtain loop descriptors; Constructing a loop according to the loop descriptor, generating loop level information and control information, and implementing loop control; Generate neural network processor executable instructions based on the loop level information and control information, so as to provide them to the neural network processor for execution.

15. The instruction processing method according to claim 14, further comprising: The neural network processor executable instructions are classified and sent to the corresponding subsequent modules.

16. The instruction processing method according to claim 14, wherein: The loop initialization instructions include: A dimension field, indicating the dimension information of the calculated data; and At least one loop initialization instruction segment, wherein each of the at least one loop initialization instruction segment corresponds to a loop level and comprises: The loop level field indicates the loop level of the current loop; The loop value field indicates the number of times the initial instruction segment of the current loop needs to loop; and The loop dimension field indicates the dimension in which the initial instruction segment of the current loop is calculated.

17. The instruction processing method according to claim 16, wherein: The loop initialization instructions also include: The dimension filling field indicates the dimension expansion information of the calculation data.

18. The instruction processing method according to claim 16, wherein: Each of the at least one loop initialization instruction segments further comprises: The space control field indicates whether the current loop performs space allocation and the direction of the space allocation.

19. The instruction processing method according to claim 16, wherein: Each of the at least one loop initialization instruction segments further comprises: The custom field is set at the start or end of the loop initial instruction segment and is configured as a custom instruction.

20. The instruction processing method according to claim 16, wherein: Each of the at least one loop initialization instruction segments further comprises: The invalid data encoding field is used to indicate the location of invalid data in the calculation data of the current cycle.

21. The instruction processing method according to claim 16, wherein: The loop initialization instructions also include: The loop initialization instruction set configuration encoding is configured to indicate the effective loop level and set the instruction set parameter bit width.

22. A chip system, comprising: An instruction processing device according to any one of claims 1 to 13; as well as A neural network processor is configured to execute the neural network processor executable instructions.

23. The chip system according to claim 22, further comprising: The processor is configured to send the loop initialization instruction to the instruction processing device.

Images