Method, apparatus and electronic device for generating instructions for neural network accelerators

Abstract

To provide an instruction generation method for a neural network accelerator, a device and an electronic device.SOLUTION: This method includes the steps of: determining a plurality of pieces of first space information corresponding to a neural network model; retrieving a policy mapping relationship from a cache on the basis of the first space information to acquire first policy information respectively corresponding to at least one piece of the first space information; and determining a global optimization policy corresponding to a neural network model on the basis of a dynamic programming algorithm and a plurality of pieces of policy information to generate an instruction that can be executed by a neural network accelerator.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present application relates to the technical field of neural networks, and in particular to a method, apparatus and electronic device for generating instructions for a neural network accelerator. [Background technology]

[0002] The neural network compiler generates an instruction sequence that can be executed on a hardware device (e.g., a Neural Network Accelerator (NNA)) based on the neural network model. As the size of the neural network model becomes larger and larger and the structure becomes more and more complex, the number of compilation methods for the neural network model also increases. When compiling, the neural network compiler can determine the optimal compilation method from multiple compilation methods so that the algorithm of the neural network model can be efficiently executed on the hardware device.

[0003] When a neural network compiler optimizes the compilation of a neural network model, since there are many compilation methods for the neural network model, it takes a long time to determine the optimal compilation method from the multiple compilation methods, which affects the user's experience of using the neural network compiler. Summary of the Invention [Problem to be solved by the invention]

[0004] The embodiments of the present disclosure provide an instruction generation method, apparatus, and electronic device for a neural network accelerator to solve the problem that the process of compiling and optimizing a neural network model takes a long time in the existing neural network compiler of the present application. [Means for solving the problem]

[0005] In an embodiment of the present disclosure, when generating an instruction sequence executable by a neural network accelerator, a plurality of first spatial information corresponding to a neural network model is fully taken into consideration, and a plurality of first policy information corresponding to the plurality of first spatial information is quickly determined by utilizing a policy mapping relationship in a cache, thereby reducing the time required for a neural network compiler to compile and optimize a neural network model.

[0006] An instruction generation method for a neural network accelerator according to an embodiment of the present disclosure includes: determining a plurality of first spatial information corresponding to a neural network model; searching a policy mapping relationship from a cache based on the first spatial information to obtain first policy information corresponding to at least one of the first spatial information; and determining a global optimization policy corresponding to the neural network model through a dynamic programming algorithm and the plurality of policy information, and generating an instruction executable by the neural network accelerator; the first spatial information is for indicating at least one neural network layer in the neural network model; The policy mapping relationship includes a plurality of target space information and target policy information corresponding to each of the target space information; The plurality of target spatial information includes at least one first spatial information; The plurality of policy information includes first policy information each corresponding to at least one of the first spatial information.

[0007] According to this technical configuration, a database is preset by setting the policy mapping relationship in advance, and optimization policies (i.e., target space information) corresponding to multiple spaces (i.e., target space information) are cached in the database. When a new neural network model needs to be compiled and optimized, the neural network model is first divided into multiple spaces (i.e., first space information), and then the optimization policies corresponding to the multiple spaces are searched for from the database (i.e., the policy mapping relationship). Once the optimization policies are searched for, calculations for the optimization policies corresponding to the spaces are not required, and the compilation optimization time of the neural network compiler is shortened.

[0008] An instruction generating device for a neural network accelerator according to one embodiment of the present disclosure includes: a partitioning module for determining a plurality of first spatial information corresponding to a neural network model; a search module for searching a policy mapping relationship in a cache based on the first spatial information to obtain first policy information corresponding to at least one of the first spatial information; and a generation module for determining a global optimization policy corresponding to the neural network model through a dynamic programming algorithm and the plurality of policy information, and generating an instruction executable by the neural network accelerator; the first spatial information is for indicating at least one neural network layer in the neural network model; The policy mapping relationship includes a plurality of target spatial information and target policy information respectively corresponding to the target spatial information, and the plurality of target spatial information includes at least one first spatial information; The plurality of policy information includes first policy information each corresponding to at least one of the first spatial information.

[0009] A computer-readable storage medium according to one aspect of an embodiment of the present disclosure stores a computer program for executing any one of the above methods.

[0010] An electronic device according to one aspect of the present disclosure includes a processor and a memory in which executable instructions for the processor are stored, and the processor realizes any one of the above methods by reading and executing the executable instructions from the memory. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is an exemplary configuration diagram of a system to which the present disclosure is applied. [Diagram 2] FIG. 1 is a structural schematic diagram of an electronic device according to an exemplary embodiment of the present disclosure. [Diagram 3] FIG. 2 is a structural schematic diagram of another electronic device according to an exemplary embodiment of the present disclosure. [Figure 4] 1 is a first schematic flowchart of an instruction generation method for a neural network accelerator according to an exemplary embodiment of the present disclosure. [Diagram 5] FIG. 2 is a structural schematic diagram of a neural network model according to an exemplary embodiment of the present disclosure. [Figure 6] 11 is a second schematic flowchart of an instruction generation method for a neural network accelerator according to an exemplary embodiment of the present disclosure. [Figure 7] 11 is a third schematic flowchart of an instruction generation method for a neural network accelerator according to an exemplary embodiment of the present disclosure. [Figure 8] 4 is a fourth schematic flowchart of an instruction generation method for a neural network accelerator according to an exemplary embodiment of the present disclosure. [Figure 9] 5 is a fifth schematic flowchart of an instruction generation method for a neural network accelerator according to an exemplary embodiment of the present disclosure. [Figure 10] 6 is a sixth schematic flowchart of an instruction generation method for a neural network accelerator according to an exemplary embodiment of the present disclosure. [Figure 11]6 is a sixth schematic flowchart of an instruction generation method for a neural network accelerator according to an exemplary embodiment of the present disclosure. [Figure 12] 6 is a sixth schematic flowchart of an instruction generation method for a neural network accelerator according to an exemplary embodiment of the present disclosure. [Figure 13] FIG. 2 is a structural diagram of an instruction generating device according to an exemplary embodiment of the present disclosure. [Figure 14] FIG. 1 is a structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, exemplary embodiments of the present application will be described in detail with reference to the drawings. The described embodiments are not all of the embodiments of the present application, but are merely a part of the embodiments of the present application, and the present application is not limited by the exemplary embodiments described herein.

[0013] The neural network compiler compiles and optimizes the neural network model so that the neural network model is efficiently executed on the hardware device. The neural network model is mainly written in a high-level language source code (e.g., Java language, Python language, etc.), and the neural network compiler compiles and optimizes the execution logic of the high-level language source code to generate machine code that can be efficiently executed on the hardware device.

[0014] The compilation optimization of the neural network compiler for the neural network model is specifically as follows: The neural network compiler traverses (also called "traversal") all the neural network layers of the neural network model, performs partitioning on all the neural network layers, obtains multiple partitioning results (also called "hierarchical space"), then determines an optimization policy corresponding to each partitioning result, and finally obtains a compilation optimization result of the neural network model based on the optimization policy corresponding to each partitioning result.

[0015] As the size of the neural network model becomes larger and the structure becomes more complex, the number of neural network layers in the neural network model also increases. Accordingly, the number of partitioning results obtained by partitioning the neural network layers also increases. The optimization policy corresponding to each partitioning result is obtained by calculation, so the more partitioning results there are, the more optimization policies the neural network compiler needs to determine. This inevitably takes a lot of time and affects the user's experience of using the neural network compiler.

[0016] In such a situation, human intervention can usually be used to reduce the partitioning results. For example, by reducing some partitioning results using a pruning technique, the process of determining an optimization policy corresponding to the reduced partitioning results is omitted, and the compilation optimization time of the neural network compiler is reduced. However, the optimization policy corresponding to the pruned partitioning results may be more compatible with the scenario in which the neural network model is applied compared to the optimization policy corresponding to the unreduced partitioning results, and the compilation optimization results obtained by the optimization policy corresponding to the unreduced partitioning results will be even worse and less than ideal.

[0017] Therefore, to solve the problem of long compilation optimization time, an embodiment of the present disclosure provides an instruction generation method for a neural network accelerator, which first divides a neural network model into a plurality of first spatial information. Then, search for policy information corresponding to the plurality of first spatial information from a policy mapping relationship, and finally use the policy information and a dynamic programming algorithm to determine a compilation optimization result of the neural network model, which is an overall optimization policy corresponding to the neural network model. The scale of the neural network model is usually very large, and the optimization policy (i.e., target policy information) corresponding to a plurality of spaces (i.e., target spatial information) can be pre-stored by using the policy mapping relationship. In this way, when the first spatial information is the same as the target spatial information, the target policy information corresponding to the target spatial information can be directly used as the first policy information corresponding to the first spatial information, thereby omitting the process of determining the first policy information and shortening the compilation optimization time of the neural network compiler.

[0018] 1 illustrates an instruction generation system according to an embodiment of the present disclosure, which includes a compiling device 11 for generating compilation instructions corresponding to a neural network model, and an executing device 12 for executing the compilation instructions provided by the compiling device 11 to perform operations on the neural network model. The operations include, but are not limited to, convolution operations, pooling operations, activation function operations, etc.

[0019] In some embodiments, the compiling device 11 may be implemented by an electronic device 20. The electronic device 20 may include, but is not limited to, a personal computer, a server computer, a multiprocessor system, a microprocessor-based system, and the like.

[0020] In some embodiments, the executing device 12 may be implemented by an electronic device 30. The electronic device 30 may include, but is not limited to, a neural network accelerator or a neural network chip.

[0021] FIG. 2 is a structural schematic diagram of an electronic device 20 according to an embodiment of the present disclosure. As shown in FIG. 2, the electronic device 20 includes at least one processor 21 and a memory 22.

[0022] The processor 21 executes the instruction generation method for the neural network accelerator in the embodiment of the present disclosure. The processor 21 can be a processor capable of supporting compilation of neural network instruction sequences, such as a high-performance central processing unit (CPU) or a graphics processing unit (GPU), or can be any other form of processing unit having compilation processing capability and / or instruction execution capability.

[0023] The memory 22 may comprise one or more computer program products including various forms of computer readable storage media such as volatile and / or non-volatile memory. The volatile memory may include, for example, random-access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. The computer readable storage medium may store one or more computer program instructions, and the processor 21 may execute the program instructions to realize the instruction generation method for a neural network accelerator described below.

[0024] 2, the electronic device 20 may further include an input device 23 and an output device 24. The input device 23 may include, for example, a keyboard, a mouse, etc., so that a compiling worker can input parameters of a neural network (e.g., an instruction sequence of a plurality of PEs, etc.). The output device 24 may include, for example, a display, a speaker, a printer, a communication network, and a remote output device connected to the communication network, etc., so that a compiling worker can check the final result of compilation or intermediate data during compilation, etc.

[0025] The embodiments of the present disclosure are not limited to a specific structure of the electronic device 20, and the electronic device 20 may include more or less components than those shown in Figure 2. For example, the electronic device 20 may include components such as a bus and an input / output interface.

[0026] FIG. 3 is a structural schematic diagram of an electronic device 30 according to an embodiment of the present disclosure. As shown in FIG. 3, the electronic device 30 may include a control unit 31 and a computing unit 32, and the control unit 31 executes instructions to control the computing unit 32 to execute the instruction generation method of the neural network accelerator described below.

[0027] The control unit 31 may include one or more processors or other forms of processing units having neural network computing capabilities and / or instruction execution capabilities. The processors may include, but are not limited to, processors capable of performing neural network related calculations, such as GPUs, brain processing units (BPUs), tensor processing units (TPUs), etc. In some embodiments, the control unit 31 controls the computation unit 32 to perform corresponding operations based on instructions of the neural network accelerator (e.g., convolution computation instructions, pooling computation instructions, activation function computation instructions, etc.).

[0028] The computation unit 32 comprises computation circuitry configured to perform computational operations of a neural network, including, but not limited to, any one or more of convolution operations, pooling operations, activation function operations, and the like.

[0029] In some embodiments, the electronic device 30 may further include a buffer memory 33 and an off-chip memory 34. The buffer memory 33 may include one or more independent cache memories or a processing unit having a data cache function, and may access the off-chip memory 34 under the control of the control unit 31. In some embodiments, the buffer memory 33 may be a high-speed, small-capacity static access memory such as a static random access memory (SRAM). Exemplarily, the buffer memory 33 may be configured to receive at least a portion of the input feature data and / or data of the convolution kernel from the off-chip memory 34 for use by the computing unit 32, and may be configured to receive and cache output feature data output by the computing unit 32, and output to the off-chip memory 34.

[0030] Off-chip memory 34 may comprise one or more independent memory or processing units with data storage capabilities, and allows memory access by buffer memory 33 under the control of control unit 31. In some embodiments, off-chip memory 34 is a slow, large capacity dynamic access memory, such as dynamic random access memory (DRAM). In some embodiments, off-chip memory 34 is double data rate (DDR).

[0031] Embodiments of the present disclosure are not limited to a specific structure of the electronic device 30, and the electronic device 30 may include more or less components than those shown in Figure 3. For example, the electronic device 30 may include components such as a bus.

[0032] The methods of the following embodiments of the present disclosure can be applied to many fields that require neural network calculations, such as image processing and intelligent driving, for example, in the field of image processing, image data can be processed by the methods according to the embodiments of the present disclosure.

[0033] FIG. 4 is an instruction generation method for a neural network accelerator according to an embodiment of the present disclosure, which can be applied to a compiling device. As shown in FIG. 4, the method includes the following steps 401 to 403.

[0034] In step 401, a plurality of first spatial information corresponding to a neural network model is determined.

[0035] Here, the first spatial information is for indicating at least one neural network layer in the neural network model.

[0036] In some embodiments, the neural network model may be a model applied in fields such as natural language processing, computer vision, speech recognition, image processing, etc. This application does not limit the usage scenario of the neural network model.

[0037] The neural network model is composed of multiple neural network layers, for example, the neural network model includes 500 or more neural network layers, but the present application does not limit the number of layers of the neural network model, and the number of layers of the neural network model is according to the actual application. FIG. 5 illustrates a neural network model 50 including a neural network layer 1, a neural network layer 2, a neural network layer 3, a neural network layer 4, and a neural network layer 5. By dividing the multiple neural network layers in the neural network model, multiple hierarchical spaces corresponding to the neural network model can be obtained, and multiple first spatial information can be obtained by processing each of the multiple hierarchical spaces. Each of the multiple first spatial information includes a neural network layer and data information of the neural network layer.

[0038] Here, the data information of the neural network layer includes the type of arithmetic operation corresponding to the neural network layer, the data size of the input data, and hyper parameters, etc. The arithmetic operation corresponding to the neural network layer includes, but is not limited to, one or more of convolution operation, pooling operation, activation function operation, etc.

[0039] The input data of the neural network layer may be image data or text data, but the present application does not limit the input data of the neural network layer. For example, the input data of the neural network layer is image data, and the data size of the image data is H×W×C. Here, H is the length of the image, W is the width of the image, and C refers to the number of channels of the image. Usually, the number of channels of a grayscale image is 1, and the number of channels of a color image is 3.

[0040] The types of arithmetic operations corresponding to different neural network layers may be the same or different. Illustratively, the types of arithmetic operations corresponding to different neural network layers are different, and the hyperparameters corresponding to different neural network layers are different accordingly. For example, when a neural network layer is a convolution layer, the corresponding hyperparameters include the size of a convolution kernel, the number of convolution kernels, the stride of convolution, and the number of zero padding. As another example, when a neural network layer is a pooling layer, the corresponding hyperparameters include the size of a pooling kernel, the stride of pooling, and the number of zero padding.

[0041] In some embodiments, as shown in FIG. 6, in the embodiment shown in FIG. 4, the above step 401 includes steps 601 to 603.

[0042] In step 601, model parameters of the neural network model are determined.

[0043] Here, the model parameters include a number of neural network layers and / or an input / output type of the neural network layers, in some embodiments, the input / output type of the neural network layers includes at least one of single-input single-output (SISO), single-input dual-output (SIDO), and dual-input dual-output (DIDO).

[0044] The neural network model is mainly written in a high-level language source code, and the optimization space corresponding to the neural network model can be obtained by converting the high-level language source code, so that the model parameters of the neural network model can be obtained and determined more intuitively through the optimization space.

[0045] Here, the conversion for the high-level language source code can be performed by inputting the high-level language source code into model visualization software (e.g., Netron), and the model visualization software can output an optimization space corresponding to the neural network model. Illustratively, the model visualization software outputs a directed acyclic graph (DAG) as shown in FIG. 5. This DAG is an optimization space. The DAG describes the neural network model in the form of a graph structure. The neural network model is abstracted into nodes and directed edges in the DAG. The nodes represent the neural network layers (i.e., neural network layer 1, neural network layer 2, neural network layer 3, neural network layer 4, and neural network layer 5) in the neural network model, and the directed edges represent the input data corresponding to the neural network layers.

[0046] The conversion to the high-level language source code can also include analyzing statements of the high-level language source code to extract multiple neural network layers in a neural network model and input data corresponding to the multiple neural network layers, and generating a calculation instruction sequence based on the extracted multiple neural network layers and the input data corresponding to the multiple neural network layers, where the calculation instruction sequence is an optimization space.

[0047] Based on the optimization space, the model parameters of the neural network model can be more graphically described. Exemplary FIG. 5 shows the optimization space described by DAG, which means that the neural network model 50 has five neural network layers, namely, neural network layer 1, neural network layer 2, neural network layer 3, neural network layer 4, and neural network layer 5. The input / output type of the neural network layer 1 is single-input single-output. The input / output type of the neural network layer 2 is also single-input single-output. The input / output type of the neural network layer 3 is single-input two-output. The input / output type of the neural network layer 4 is single-input single-output. The input / output type of the neural network layer 5 is two-input single-output (DISO).

[0048] The input / output type of the neural network layer mainly relates to the neural network layer in the neural network model, and the input / output type of the neural network model may be three inputs and three outputs or four inputs and four outputs, whereas the present application is not limited thereto.

[0049] In step 602, a number of hierarchical spaces corresponding to neural network layers in the neural network model are obtained based on model parameters of the neural network model.

[0050] According to the model parameters of the neural network model, the neural network layers in the neural network model are divided to obtain a plurality of hierarchical spaces. Each hierarchical space includes at least one neural network layer. Exemplarily, the division of the neural network layers in the neural network model may be performed according to a preset rule. Here, the preset rule may be related to the number of neural network layers, may be related to the input / output type of the neural network layers, or may be related to both the number of neural network layers and the input / output type of the neural network. The preset rule is not limited in the present application.

[0051] For example, the preset rule can divide two neural network layers into one hierarchical space to obtain multiple hierarchical spaces. Combined with FIG. 5, there are five hierarchical spaces after division, which are hierarchical space a (neural network layer 1 and neural network layer 2), hierarchical space b (neural network layer 2 and neural network layer 3), hierarchical space c (neural network layer 3 and neural network layer 4), hierarchical space d (neural network layer 4 and neural network layer 5), and hierarchical space e (neural network layer 3 and neural network layer 5).

[0052] Alternatively, the preset rule can divide the three neural network layers into one hierarchical space to obtain multiple hierarchical spaces. Combined with FIG. 5, there are four hierarchical spaces after division, which are hierarchical space f (neural network layer 1 to neural network layer 3), hierarchical space g (neural network layer 2 to neural network layer 4), hierarchical space h (neural network layer 3 to neural network layer 5), and hierarchical space i (neural network layer 2, neural network layer 3, and neural network layer 5).

[0053] Alternatively, the preset rule can be divided according to the input / output type of the neural network layer to obtain multiple hierarchical spaces. Combined with FIG. 5, there are four hierarchical spaces after division, which are hierarchical space h (neural network layer 1 and neural network layer 2), hierarchical space l (neural network layer 3), hierarchical space j (neural network layer 4), and hierarchical space k (neural network layer 5). Here, the input / output type of the neural network layer 1 and the neural network layer 2 is single-input single-output, the input / output type of the neural network layer 3 is single-input two-output, the input / output type of the neural network 4 is single-input single-output, and the input / output type of the neural network layer 5 is two-input single-output. The input / output type of the neural network layer 4 is the same as that of the neural network layer 1 and the neural network layer 2, but since the input data of the neural network layer 4 depends on the output data of the neural network 3, the neural network 4 cannot be divided into the hierarchical space h, but needs to be separately divided into the hierarchical space j.

[0054] Taking the neural network model 50 shown in FIG. 5 as an example, the neural network model 50 can be divided into a plurality of hierarchical spaces, the details of which are shown in Table 1 below. [Table 1]

[0055] According to Table 1, the neural network model 50 can be divided into 18 hierarchical spaces, which are hierarchical space 1, hierarchical space 2, hierarchical space 3, hierarchical space 4, hierarchical space 5, hierarchical space 6, hierarchical space 7, hierarchical space 8, hierarchical space 9, hierarchical space 10, hierarchical space 11, hierarchical space 12, hierarchical space 13, hierarchical space 14, hierarchical space 15, hierarchical space 16, hierarchical space 17, and hierarchical space 18, respectively.

[0056] Here, hierarchical space 1 is composed of neural network layer 1. Hierarchical space 2 is composed of neural network layer 2. Hierarchical space 3 is composed of neural network layer 3. Hierarchical space 4 is composed of neural network layer 4. Hierarchical space 5 is composed of neural network layer 5. Hierarchical space 6 is composed of neural network layers 1-2 (specifically, including neural network layer 1 and neural network layer 2). Hierarchical space 7 is composed of neural network layers 2-3 (specifically, including neural network layer 2 and neural network layer 3). Hierarchical space 8 is composed of neural network layers 3-4 (specifically, including neural network layer 3 and neural network layer 4). Hierarchical space 9 is composed of neural network layers 4-5 (specifically, including neural network layer 4 and neural network layer 5). Hierarchical space 10 is composed of neural network layers 3 and 5 (specifically, including neural network layer 3 and neural network layer 5). Hierarchical space 11 is composed of neural network layers 1 to 3 (specifically, including neural network layer 1, neural network layer 2, and neural network layer 3). Hierarchical space 12 is composed of neural network layers 2 to 4 (specifically, including neural network layer 2, neural network layer 3, and neural network layer 4). Hierarchical space 13 is composed of neural network layers 3 to 5 (specifically, including neural network layer 3, neural network layer 4, and neural network layer 5). Hierarchical space 14 is composed of neural network layers 2 to 3 and 5 (specifically, including neural network layer 2, neural network layer 3, and neural network layer 5). Hierarchical space 15 is composed of neural network layers 1 to 4 (specifically, including neural network layer 1, neural network layer 2, neural network layer 3, and neural network layer 4).Hierarchical space 16 is composed of neural network layers 2 to 5 (specifically, including neural network layer 2, neural network layer 3, neural network layer 4, and neural network layer 5). Hierarchical space 17 is composed of neural network layers 1 to 3 and 5 (specifically, including neural network layer 1, neural network layer 2, neural network layer 3, and neural network layer 5). Hierarchical space 18 is composed of neural network layers 1 to 5 (specifically, including neural network layer 1, neural network layer 2, neural network layer 3, neural network layer 4, and neural network layer 5).

[0057] In step 603, a plurality of first spatial information is obtained based on a plurality of hierarchical spaces.

[0058] In some embodiments, obtaining a plurality of first spatial information based on a plurality of hierarchical spaces can specifically involve encoding (coding) based on the plurality of hierarchical spaces to obtain a plurality of first spatial information, and the plurality of hierarchical spaces have a one-to-one correspondence with the plurality of first spaces.

[0059] In some embodiments, as shown in FIG. 7, in the embodiment shown in FIG.

[0060] In step 701, a plurality of hierarchical spaces and hardware resource information of a neural network accelerator are encoded to obtain a plurality of first space information.

[0061] Since the neural network model compiled and optimized by the neural network encoder is executed on a hardware device, after obtaining the plurality of hierarchical spaces corresponding to the neural network model, the hardware device on which the neural network model is executed must also be considered. Therefore, after determining the hardware device on which the neural network model is executed, the plurality of hierarchical spaces corresponding to the neural network model and the hardware resource information of the hardware device are combined and encoded. For example, when it is determined that the neural network model is executed on a neural network accelerator, each hierarchical space in the plurality of hierarchical spaces and the hardware resource information of the neural network accelerator are encoded respectively.

[0062] In some embodiments, the encoding of the hardware resource information of the plurality of hierarchical spaces and the neural network accelerator to obtain the first spatial information may include encoding the hardware resource information of the plurality of hierarchical spaces and the neural network accelerator by a hash algorithm to obtain the first spatial information. The encoded plurality of first spatial information may be represented by a plurality of key values. The plurality of key values ​​correspond one-to-one to the plurality of hierarchical spaces, and the plurality of key values ​​uniquely identify the plurality of hierarchical spaces and the neural network accelerator.

[0063] Also, during the encoding process, there can be multiple mapping relationships, and the processor stores the multiple mapping relationships. Wherein, the mapping relationships are for indicating the first spatial information and the hierarchical space corresponding to the first spatial information and the hardware resource information of the neural network accelerator. The hierarchical space stored by the multiple mapping relationships is different. The first spatial information stored by the multiple mapping relationships is also different.

[0064] Here, the coding of each hierarchical space in the plurality of hierarchical spaces and the hardware resource information of the neural network accelerator may use other algorithms as long as a unique identifier that characterizes the hierarchical space and the neural network accelerator can be obtained, and the present application does not limit the coding algorithm of the hierarchical space and the neural network accelerator. The plurality of first spatial information after coding occupies a smaller storage space than the hierarchical space and the hardware resource information of the neural network accelerator before coding. In addition, compared with the case where the hierarchical space and the hardware resource information of the neural network accelerator before coding are used for data processing, the data resources processed by using the plurality of first spatial information are less, and the hardware resources required for processing the data resources are also less. Therefore, the use of the coding method can reduce the consumption of hardware resources to a certain extent.

[0065] In step 402, search for a policy mapping relationship from a cache based on the first spatial information to obtain first policy information respectively corresponding to at least one first spatial information.

[0066] Here, the policy mapping relationship includes a plurality of pieces of target space information and target policy information respectively corresponding to the target space information, and the plurality of pieces of target space information includes at least one piece of first space information.

[0067] As can be seen from steps 601 to 701, when the size of the neural network model becomes larger (i.e., the number of neural network layers increases), the hierarchical space corresponding to the neural network model becomes larger, and the first spatial information corresponding to the hierarchical space becomes larger. Next, the first policy information corresponding to each first spatial information is determined, respectively, and finally, a global optimization policy corresponding to the neural network model is obtained based on the plurality of first policy information. The first policy information of the plurality of first spatial information can be obtained only by calculation. Therefore, the more the first spatial information, the more the first policy information corresponding to the first spatial information that needs to be determined. This significantly increases the amount of calculation and the time required for calculation.

[0068] Therefore, the present application uses the policy mapping relationship to reduce the process of determining the first policy information corresponding to the first spatial information, thereby reducing the consumption of time and hardware resources. Exemplarily, a large amount of target spatial information and target policy information corresponding to each target spatial information are stored in advance in the policy mapping relationship. Here, the target spatial information is stored in the form of a key value, and the target policy information corresponding to the target spatial information is stored in the form of a value value. After determining the plurality of first spatial information corresponding to the neural network model, the plurality of first spatial information is also stored in the form of a key value, so that the key value corresponding to the plurality of first spatial information and the key value corresponding to the plurality of target spatial information in the policy mapping relationship can be directly compared. After determining the target spatial information that matches the first spatial information from the plurality of target spatial information by the comparison, the target policy information corresponding to the target spatial information is set as the first policy information corresponding to the first spatial information. As a result, it is not necessary to determine the first policy information corresponding to the first spatial information again for the first spatial information that matches the target spatial information, and the first policy information can be used as it is, thereby reducing the time for compilation optimization.

[0069] In step 403, a global optimization policy corresponding to the neural network model is determined by a dynamic programming algorithm and a plurality of policy information, and an instruction executable by the neural network accelerator is generated.

[0070] Here, the plurality of policy information includes first policy information each corresponding to at least one of the first spatial information.

[0071] The dynamic programming algorithm analyzes multiple pieces of policy information, determines from the multiple pieces of policy information a policy that is more suitable for a scenario to which the neural network model is applied, sets this policy as a global optimization policy corresponding to the neural network model, and generates instructions executable by the neural network accelerator based on this global optimization policy.

[0072] For example, a scenario to which a neural network model is applied requires that the neural network model outputs data in a short time. In this case, the dynamic programming algorithm analyzes multiple policy information and searches for a policy with the shortest time required from the multiple policy information to obtain a global optimization policy. As another example, a scenario to which a neural network model is applied requires that the neural network model occupies less memory access bandwidth resources within the same time period, which is better. In this case, the dynamic programming algorithm analyzes multiple policy information and searches for a policy with the least memory access bandwidth resources occupied from the multiple policy information to obtain a global optimization policy.

[0073] According to this technical configuration, a database is preset by setting the policy mapping relationship in advance, and optimization policies (i.e., target space information) corresponding to multiple spaces (i.e., target space information) are cached in the database. When a new neural network model needs to be compiled and optimized, the neural network model is divided into multiple spaces (i.e., first space information), and then the optimization policies corresponding to the multiple spaces are searched for from the database (i.e., the policy mapping relationship). Once the optimization policies are searched for, calculations for the optimization policies corresponding to the spaces are not required, and the compilation optimization time of the neural network compiler is shortened.

[0074] In some embodiments, as shown in FIG. 8, on top of the embodiment shown in FIG. 4 above, the method further includes step 801.

[0075] In step 801, when at least one piece of second spatial information different from the target spatial information is present in the plurality of pieces of first spatial information, second policy information corresponding to each piece of at least one piece of second spatial information is determined.

[0076] Here, the plurality of policy information includes second policy information each corresponding to at least one of the second spatial information.

[0077] The target spatial information that matches the plurality of pieces of first spatial information is searched from the policy mapping relationship, and when all the target spatial information corresponding to the plurality of pieces of first spatial information is searched, the target policy information corresponding to each of the searched plurality of pieces of first spatial information is set as the first policy information corresponding to each of the first spatial information in the plurality of pieces of first spatial information. As a result, the plurality of policy information in step 403 becomes the first policy information corresponding to each of the first spatial information in the plurality of pieces of first spatial information.

[0078] When only target spatial information corresponding to a portion of the first spatial information is searched for among the plurality of first spatial information, the target policy information corresponding to each of the searched target spatial information is set as the first policy information corresponding to the portion of the first spatial information.

[0079] The first spatial information for which the target spatial information is not found is determined as the second spatial information (i.e., at least one second spatial information different from the target spatial information, which exists among the plurality of first spatial information), and the second policy information corresponding to each second spatial information is determined in sequence. That is, the plurality of policy information includes the first policy information corresponding to a part of the first spatial information, and the plurality of second policy information corresponding to the plurality of second spatial information.

[0080] In some embodiments, as shown in FIG. 9, in the embodiment shown in FIG. 4 above, the step of "determining second policy information corresponding to at least one piece of second spatial information" in the above step 801 "if at least one piece of second spatial information different from the target spatial information exists in the multiple pieces of first spatial information, determine second policy information corresponding to at least one piece of second spatial information" includes steps 901 to 903.

[0081] In step 901, based on the second spatial information, a number of candidate policies corresponding to the second spatial information are determined.

[0082] Since the first spatial information is obtained by encoding a hierarchical space, at least one second spatial information in the plurality of first spatial information is also obtained by encoding. As can be seen from step 701, during the encoding process, the processor stores a plurality of mapping relationships. Since the second spatial information is obtained by encoding, a second space corresponding to the second spatial information can be searched from the plurality of mapping relationships. The second space includes at least one neural network layer and data information of the neural network layer. The second space is a part of a plurality of hierarchical spaces.

[0083] For details of the data information of the neural network layer, please refer to the explanation of step 401, and the explanation will be omitted here.

[0084] Determining a plurality of candidate policies corresponding to the second space information specifically means determining a plurality of candidate policies corresponding to the neural network layer in the second space. The determination process determines the data size of the input data of the neural network layer in the second space according to the data information of the neural network layer in the second space, and then obtains a plurality of candidate policies according to the data size of the input data and the hardware resources of the neural network accelerator.

[0085] In step 902, a plurality of computing nodes determine a plurality of cost parameters for executing a plurality of candidate policies corresponding to the second spatial information.

[0086] Since one piece of second spatial information can correspond to multiple candidate policies, in order to quickly determine multiple cost parameters of multiple candidate policies corresponding to the at least one piece of second spatial information, the at least one piece of second spatial information can be assigned to multiple computing nodes, and the computing nodes can be utilized to determine multiple cost parameters of multiple candidate policies corresponding to the second spatial information. An assignment rule for assigning the second spatial information to the computing nodes is that the computing resources required to process the multiple cost parameters of the multiple candidate policies corresponding to the second spatial information are less than the currently remaining computing resources of the computing nodes.

[0087] Therefore, the neural network compiler obtains the load capacity, the data to be processed, and the remaining computational resources of each computing node in the processor (e.g., CPU), and then estimates the computational resources required to process the second spatial information, and compares the remaining computational resources of the computing node with the computational resources required to process the second spatial information to determine the computing node for processing the second spatial information. During the process of determining the computing node, the load balance between each computing node is also taken into consideration, so that the overload operation of a single computing node can be avoided.

[0088] A plurality of computing nodes executes a plurality of candidate policies corresponding to each second space, respectively, to obtain a plurality of cost parameters corresponding to each candidate policy. This speeds up the processing and shortens the time required. Here, the plurality of cost parameters include processing time, consumed computing resources, occupied storage resources, and processing cycles.

[0089] In step 903, second policy information corresponding to the second spatial information is determined from the plurality of candidate policies based on the plurality of cost parameters.

[0090] The application scenario of the neural network model and the multiple cost parameters are combined to determine second policy information corresponding to the second spatial information from the multiple candidate policies. For example, the application scenario of the neural network model is real-time navigation of a car. The multiple candidate policies corresponding to the second spatial information include candidate policy 1 and candidate policy 2. According to the cost parameter corresponding to candidate policy 1, the candidate policy 1 requires a long time to perform the calculation of the neural network layer corresponding to the second spatial information, but occupies a small bandwidth resource. According to the cost parameter corresponding to candidate policy 2, the candidate policy 2 requires a short time to perform the calculation of the neural network layer corresponding to the second spatial information, but occupies a large bandwidth resource. In order to be able to receive the navigation result in real time during driving, in this scenario, the candidate policy 2 is taken as the second policy information.

[0091] In some embodiments, as shown in FIG. 10, in the embodiment shown in FIG.

[0092] In step 1001, a second optimization policy is determined from a number of candidate policies based on a number of cost parameters.

[0093] In step 1002, a second optimization policy is encoded to obtain second policy information corresponding to the second spatial information.

[0094] According to step 903, a second optimization policy can be determined from a plurality of candidate policies based on the application scenario of the neural network model and a plurality of cost parameters, and then the second optimization policy can be encoded to obtain second policy information. The second policy information can be represented by a value. Since the second optimization policy has a lot of data information, the amount of data of the second optimization policy can be reduced by encoding it. This can not only reduce the memory load, but also reduce the calculation load.

[0095] In some embodiments, as shown in FIG. 11, above the embodiment shown in FIG.

[0096] In step 1101, second policy information corresponding to the second spatial information is added to the policy mapping relationship.

[0097] As can be seen from step 801, there is a possibility that at least one second spatial information different from the target spatial information exists in the plurality of first spatial information, so the computing node determines the second policy information corresponding to the second spatial information, and then the second spatial information and the second policy information corresponding to the second spatial information can be updated to the policy mapping relationship. This allows the amount of data in the policy mapping relationship to be expanded, and the next time compilation optimization is performed, the updated policy mapping relationship can be used for searching, the search range is expanded, the possibility of searching for the first policy information is increased, and the time required for compilation optimization is further reduced.

[0098] In some embodiments, as shown in FIG. 12, in the embodiment shown in FIG. 11, the above step 403 includes steps 1201 to 1203.

[0099] In step 1201, a first optimization policy corresponding to the first policy information and a second optimization policy corresponding to the second policy information are determined based on the first policy information and the second policy information.

[0100] Since the first policy information and the second policy information are generated by encoding, the multiple mapping relationships stored by the processor include the first policy information and the second policy information. When it is necessary to determine a global optimization policy corresponding to the neural network model based on the first policy information and the second policy information, the first optimization policy corresponding to the first policy information and the second optimization policy corresponding to the second policy information can be searched for based on the multiple mapping relationships.

[0101] In step 1202, a global optimization policy corresponding to the neural network model is determined from the at least one first optimization policy and the at least one second optimization policy according to a dynamic programming algorithm.

[0102] Taking the neural network model 50 shown in FIG. 5 as an example, as shown in Table 1, the neural network model 50 corresponds to 18 hierarchical spaces. A search is performed on the policy mapping relationship to determine that the policy mapping relationship only includes a plurality of hierarchical spaces corresponding to neural network layer 1 to neural network layer 3 and first optimization policies corresponding to the plurality of hierarchical spaces, respectively. The plurality of hierarchical spaces are hierarchical space 1, hierarchical space 2, hierarchical space 3, hierarchical space 6, hierarchical space 7, and hierarchical space 11, respectively. The first optimization policies corresponding to the plurality of hierarchical spaces, respectively, are the first optimization policy corresponding to hierarchical space 1, the first optimization policy corresponding to hierarchical space 2, the first optimization policy corresponding to hierarchical space 3, the first optimization policy corresponding to hierarchical space 6, the first optimization policy corresponding to hierarchical space 7, and the first optimization policy corresponding to hierarchical space 11.

[0103] Since it is determined that the policy mapping relationship includes only multiple hierarchical spaces corresponding to neural network layer 1 to neural network layer 3 and first optimization policies corresponding to each of the multiple hierarchical spaces, it is necessary to newly determine multiple hierarchical spaces related to neural network layer 4 or neural network layer 5 and second optimization policies corresponding to each of the multiple hierarchical spaces.

[0104] Exemplarily, steps 901 to 1002 can be used to determine a plurality of hierarchical spaces associated with neural network layer 4 or neural network layer 5 and second optimization policies corresponding to the plurality of hierarchical spaces, respectively. The plurality of hierarchical spaces associated with neural network layer 4 or neural network layer 5 are hierarchical space 4, hierarchical space 5, hierarchical space 8, hierarchical space 9, hierarchical space 10, hierarchical space 12, hierarchical space 13, hierarchical space 14, hierarchical space 15, hierarchical space 16, hierarchical space 17, and hierarchical space 18, respectively. The second optimization policies corresponding to the multiple hierarchical spaces associated with neural network layer 4 or neural network layer 5 are: a second optimization policy corresponding to hierarchical space 4, a second optimization policy corresponding to hierarchical space 5, a second optimization policy corresponding to hierarchical space 8, a second optimization policy corresponding to hierarchical space 9, a second optimization policy corresponding to hierarchical space 10, a second optimization policy corresponding to hierarchical space 12, a second optimization policy corresponding to hierarchical space 13, a second optimization policy corresponding to hierarchical space 14, a second optimization policy corresponding to hierarchical space 15, a second optimization policy corresponding to hierarchical space 16, a second optimization policy corresponding to hierarchical space 17, and a second optimization policy corresponding to hierarchical space 18.

[0105] According to the dynamic programming algorithm, based on the first optimization policy corresponding to hierarchical space 1, the first optimization policy corresponding to hierarchical space 2, the first optimization policy corresponding to hierarchical space 3, the first optimization policy corresponding to hierarchical space 6, the first optimization policy corresponding to hierarchical space 7, the first optimization policy corresponding to hierarchical space 11, the second optimization policy corresponding to hierarchical space 4, the second optimization policy corresponding to hierarchical space 5, the second optimization policy corresponding to hierarchical space 8, the second optimization policy corresponding to hierarchical space 9, the second optimization policy corresponding to hierarchical space 10, the second optimization policy corresponding to hierarchical space 12, the second optimization policy corresponding to hierarchical space 13, the second optimization policy corresponding to hierarchical space 14, the second optimization policy corresponding to hierarchical space 15, the second optimization policy corresponding to hierarchical space 16, the second optimization policy corresponding to hierarchical space 17, and the second optimization policy corresponding to hierarchical space 18, it is determined that the overall optimization policy corresponding to the neural network model 50 is the first optimization policy corresponding to hierarchical space 11, the second optimization policy corresponding to hierarchical space 4, and the second optimization policy corresponding to hierarchical space 5.

[0106] In step 1203, instructions executable by the neural network accelerator are generated based on the global optimization policy corresponding to the neural network model.

[0107] In the embodiment of the present disclosure, when generating instructions executable by a neural network accelerator, the compiling device divides the neural network model into a plurality of first spatial information, then searches for a plurality of first policy information corresponding to the plurality of first spatial information from the policy mapping relationship, and if there is any of the plurality of first spatial information that is not searched from the policy mapping relationship, determines the first policy information corresponding to the not-searched spatial information (i.e., the second spatial information), and then uses the plurality of policy information (i.e., the first policy information and the second policy information) and a dynamic programming algorithm to determine a compilation optimization result of the neural network model, which is an overall optimization policy corresponding to the neural network model. Since the scale of the neural network model is usually very large, optimization policies (i.e., target policy information) corresponding to several spaces (i.e., target spatial information) can be pre-stored in the manner of the policy mapping relationship. Thus, when the first spatial information is the same as the target spatial information, the target policy information corresponding to the target spatial information can be directly taken as the first policy information corresponding to the first spatial information, thereby omitting the process of determining the first policy information and shortening the compilation optimization time of the neural network compiler.

[0108] FIG. 13 illustrates an instruction generating device according to an embodiment of the present disclosure. As shown in FIG. 13, an instruction generating device 1300 includes a division module 1301, a search module 1302, and a generation module 1303.

[0109] The partition module 1301 determines a plurality of first spatial information corresponding to the neural network model, the first spatial information being for indicating at least one neural network layer in the neural network model.

[0110] The search module 1302 searches for a policy mapping relationship from a cache based on the first spatial information to obtain first policy information corresponding to the at least one first spatial information, where the policy mapping relationship includes a plurality of target spatial information and target policy information respectively corresponding to the target spatial information, and the plurality of target spatial information includes the at least one first spatial information.

[0111] The generation module 1303 determines a global optimization policy corresponding to the neural network model through a dynamic programming algorithm and multiple policy information, and generates instructions executable by the neural network accelerator, where the multiple policy information includes first policy information each corresponding to at least one first spatial information.

[0112] In some embodiments, the instruction generating device 1300 further includes a determining module, which determines second policy information corresponding to the at least one second space information, respectively, when the plurality of first space information includes at least one second space information different from the target space information, and the plurality of policy information includes the second policy information corresponding to the at least one second space information, respectively.

[0113] In some embodiments, the determination module includes a first determination unit, a second determination unit, and a third determination unit, where the first determination unit determines, based on the second spatial information, a plurality of candidate policies corresponding to the second spatial information, the second determination unit determines a plurality of cost parameters for a computing node to execute the plurality of candidate policies corresponding to the second spatial information, and the third determination unit determines, based on the plurality of cost parameters, second policy information corresponding to the second spatial information from the plurality of candidate policies.

[0114] In some embodiments, the third determination unit further determines a second optimization policy from the multiple candidate policies based on the multiple cost parameters, and encodes the second optimization policy to obtain a second policy corresponding to the second spatial information.

[0115] In some embodiments, the instruction generating device 1300 comprises an adding module for adding second policy information corresponding to the second spatial information to the policy mapping relationship.

[0116] In some embodiments, the partitioning module 1301 further determines model parameters of the neural network model; obtains a plurality of hierarchical spaces corresponding to neural network layers in the neural network model based on the model parameters of the neural network model; and obtains a plurality of first spatial information based on the plurality of hierarchical spaces, where the model parameters include the number of neural network layers and / or the input / output types of the neural network layers.

[0117] In some embodiments, the input / output type of the neural network layer includes at least one of single-input single-output, single-input two-output, and two-input two-output, and the model parameters are represented as a directed acyclic graph or a computational instruction sequence.

[0118] In some embodiments, the partition module 1301 further encodes the multiple hierarchical spaces and hardware resource information of the neural network accelerator to obtain the first spatial information.

[0119] In some embodiments, the generation module 1303 further determines, based on the first policy information and the second policy information, a first optimization policy corresponding to the first policy information and a second optimization policy corresponding to the second policy information, determines an optimization policy corresponding to the neural network model from the at least one first optimization policy and the at least one second optimization policy according to a dynamic programming algorithm, and generates instructions executable by the neural network accelerator based on the optimization policy corresponding to the neural network model.

[0120] FIG. 14 is a structural diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 14, an electronic device 1400 includes one or more processors 1401 and a memory 1402 .

[0121] The processor 1401 may be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities, and may control other components within the electronic device 1400 to perform desired functions.

[0122] The memory 1402 may comprise one or more computer program products including various forms of computer readable storage media such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache). The non-volatile memory may include, for example, a read only memory (ROM), a hard disk, a flash memory, etc. The computer readable storage medium may store one or more computer program instructions, and the processor 1401 may execute the program instructions to realize the instruction generation method of each embodiment of the present disclosure described above and / or other desired functions.

[0123] In one example, the electronic device 1400 may further comprise an input device 1403 and an output device 1404 connected to each other via a bus system and / or other form of connection (not shown).

[0124] For simplicity, Fig. 14 shows only some of the components related to the present disclosure in the electronic device 1400, and omits components such as buses and input / output interfaces. Furthermore, depending on specific application conditions, the electronic device 1400 may further include any other suitable components.

[0125] In addition to the methods and devices described above, embodiments of the present disclosure may be a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform steps in the instruction generation method according to the embodiments described in the exemplary method section of the present disclosure.

[0126] The computer program product may be written with program code for carrying out operations of the embodiments of the present disclosure in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java or C++, and traditional procedural programming languages ​​such as "C" or similar. The program code may be executed entirely on the user's computing device, partially on the user's device, as a separate software package, partly on the user's computing device and partly on a remote computing device, or entirely on a remote computing device or a server.

[0127] An embodiment of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps of the instruction generation method according to each embodiment described in the above exemplary method of the present disclosure.

[0128] The computer readable storage medium may use any combination of one or more readable media. The readable storage medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include an electrical connection with one or more conductors, a mobile hard drive, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0129] Although the basic principles of the present disclosure have been described above with reference to specific embodiments, the advantages, advantages, effects, etc. mentioned in the present disclosure are not limited but merely illustrative, and these advantages, advantages, effects, etc. are not necessarily possessed by each embodiment of the present disclosure. In addition, the specific details disclosed above are not limited but merely serve to serve as examples and to facilitate understanding, and the above details do not necessarily limit the present disclosure to be realized by the above specific details.

[0130] The foregoing description has been provided for purposes of illustration and description, and is not intended to limit the embodiments of the present disclosure to the form disclosed herein. While a number of exemplary aspects and embodiments have been described in the foregoing description, those skilled in the art may recognize certain variations, modifications, variations, additions, and subcombinations thereof.

Claims

1. A method for generating instructions for a neural network accelerator, each step of which is executed by an instruction generating device for a neural network accelerator, comprising: determining a first plurality of spatial information corresponding to the neural network model; Retrieving a policy mapping relationship from a cache based on the first spatial information to obtain first policy information respectively corresponding to at least one first spatial information; determining a global optimization policy corresponding to the neural network model through a dynamic programming algorithm and a plurality of policy information, and generating instructions executable by a neural network accelerator; the first spatial information is for indicating at least one neural network layer in the neural network model; The policy mapping relationship includes a plurality of target spatial information to be compared with the plurality of first spatial information, and target policy information corresponding to each of the target spatial information; the plurality of target spatial information includes target spatial information that matches the at least one first spatial information; 11. A method for generating instructions for a neural network accelerator, comprising: the plurality of pieces of policy information including first policy information each corresponding to the at least one piece of first spatial information.

2. The instruction generation method further includes, when at least one second spatial information different from the target spatial information is present in the plurality of first spatial information, determining second policy information corresponding to each of the at least one second spatial information; 2. The method of claim 1, wherein the plurality of policy information includes second policy information each corresponding to the at least one second spatial information.

3. The step of determining second policy information corresponding to each of the at least one second spatial information includes: determining, based on the second spatial information, a number of candidate policies corresponding to the second spatial information; determining a number of cost parameters for a number of computing nodes to execute a number of candidate policies corresponding to the second spatial information; and determining the second policy information corresponding to the second spatial information from the plurality of candidate policies based on the plurality of cost parameters.

4. The step of determining the second policy information corresponding to the second spatial information from the plurality of candidate policies based on the plurality of cost parameters includes: determining a second optimization policy from the plurality of candidate policies based on the plurality of cost parameters; and encoding the second optimization policy to obtain the second policy information corresponding to the second spatial information.

5. 5. The method for generating instructions for a neural network accelerator according to claim 2, further comprising the step of adding second policy information corresponding to the second spatial information to the policy mapping relationship.

6. The step of determining a plurality of first spatial information corresponding to the neural network model includes: determining model parameters of the neural network model; obtaining a plurality of hierarchical spaces corresponding to neural network layers in the neural network model based on model parameters of the neural network model; and acquiring a plurality of first spatial information based on the plurality of hierarchical spaces; The method for generating instructions for a neural network accelerator according to any one of claims 1 to 4, characterized in that the model parameters include a number of neural network layers and / or input / output types of the neural network layers.

7. an input / output type of the neural network layer includes at least one of a single-input / single-output, a single-input / two-output, and a two-input / two-output; 7. The method of claim 6, wherein the model parameters are represented as a directed acyclic graph or a computational instruction sequence.

8. The step of acquiring a plurality of pieces of first spatial information based on the plurality of hierarchical spaces includes:

7. The method for generating instructions for a neural network accelerator according to claim 6, further comprising: encoding the plurality of hierarchical spaces and hardware resource information of the neural network accelerator to obtain a plurality of the first spatial information.

9. determining a global optimization policy corresponding to the neural network model using a dynamic programming algorithm and a plurality of policy information to generate instructions executable by a neural network accelerator, determining, based on the first policy information and the second policy information, a first optimization policy corresponding to the first policy information and a second optimization policy corresponding to the second policy information; determining an overall optimization policy corresponding to the neural network model from at least one of the first optimization policies and at least one of the second optimization policies according to a dynamic programming algorithm; and generating instructions executable by the neural network accelerator based on a global optimization policy corresponding to the neural network model.

10. a partition module for determining a plurality of first spatial information corresponding to the neural network model; a search module for searching a policy mapping relationship in a cache based on the first spatial information to obtain first policy information corresponding to at least one of the first spatial information; a generation module for determining a global optimization policy corresponding to the neural network model through a dynamic programming algorithm and a plurality of policy information, and generating an instruction executable by a neural network accelerator; the first spatial information is for indicating at least one neural network layer in the neural network model; The policy mapping relationship includes a plurality of target spatial information to be compared with the plurality of first spatial information, and target policy information corresponding to each of the target spatial information; the plurality of target spatial information includes target spatial information that matches the at least one first spatial information; The instruction generating device for a neural network accelerator, wherein the plurality of pieces of policy information include first policy information each corresponding to the at least one piece of first spatial information.

11. A computer-readable storage medium storing a computer program for executing the method for generating instructions for a neural network accelerator according to any one of claims 1 to 4.

12. A processor; a memory for storing instructions executable by the processor; The electronic device is characterized in that the processor realizes the instruction generation method for a neural network accelerator according to any one of claims 1 to 4 by reading the executable instructions from the memory and executing them.

Images