Instruction generation method and apparatus for neural network accelerator, and electronic device

Abstract

A method and device for generating instructions for a neural network accelerator and an electronic device are disclosed, relating to the technical field of neural networks, the method comprising determining a plurality of first spatial information corresponding to a neural network model, the first spatial information being used to represent at least one neural network layer in the neural network model; according to the first spatial information, searching for a strategy mapping relationship in a cache to obtain first strategy information corresponding to at least one first spatial information; the strategy mapping relationship comprises a plurality of target spatial information and target strategy information corresponding to each target spatial information, and the plurality of target spatial information comprises at least one first spatial information; based on a dynamic programming algorithm and a plurality of strategy information, determining a total optimization strategy corresponding to the neural network model to generate instructions executable by the neural network accelerator, so as to reduce the time length of the neural network compiler compiling and optimizing the neural network model.

Description

Technical Field

[0001] This application relates to the field of neural network technology, and in particular to an instruction generation method, apparatus and electronic device for neural network accelerators. Background Technology

[0002] A neural network compiler is used to generate a sequence of instructions executable by hardware devices (e.g., neural network accelerators) based on a neural network model. As neural network models become larger and more complex, the methods for compiling them also increase. During compilation, the neural network compiler can determine the optimal compilation method from various options to enable the hardware device to efficiently execute the neural network model's algorithm.

[0003] When a neural network compiler optimizes a neural network model, the numerous compilation methods available for the model can lead to a lengthy process in determining the optimal method, negatively impacting the user experience. Summary of the Invention

[0004] Existing neural network compilers take a long time to compile and optimize neural network models.

[0005] To address the aforementioned technical problems, this disclosure is proposed. Embodiments of this disclosure provide an instruction generation method, apparatus, and electronic device. When generating an executable instruction sequence for a neural network accelerator, the method fully considers multiple first spatial information corresponding to the neural network model and utilizes policy mapping relationships in a cache to quickly determine multiple first policy information corresponding to the multiple first spatial information, thereby reducing the time required for the neural network compiler to compile and optimize the neural network model.

[0006] According to one aspect of this disclosure, a method for generating instructions for a neural network accelerator is provided, comprising: first, determining multiple first spatial information corresponding to a neural network model, wherein the first spatial information represents at least one neural network layer in the neural network model; second, searching a policy mapping relationship in a cache based on the first spatial information to obtain first policy information corresponding to at least one first spatial information; the policy mapping relationship includes multiple target spatial information and target policy information corresponding to each target spatial information, wherein the multiple target spatial information includes at least one first spatial information; and finally, determining a total optimization policy corresponding to the neural network model based on a dynamic programming algorithm and the multiple policy information to generate instructions executable by the neural network accelerator; wherein the multiple policy information includes first policy information corresponding to at least one first spatial information.

[0007] Based on this scheme, by pre-setting the policy mapping relationship, it is equivalent to setting up a database in advance, which caches the optimization policies (i.e., target policy information) corresponding to multiple spaces (i.e., target space information). When it is necessary to compile and optimize a new neural network model, the neural network model is first split into multiple spaces (i.e., first space information), and then the optimization policies corresponding to multiple spaces are searched from the database (i.e., policy mapping relationship). If they can be found, it is not necessary to calculate the optimization policy corresponding to the space, thus directly reducing the compilation and optimization time of the neural network compiler.

[0008] According to one aspect of the present disclosure, an instruction generation apparatus for a neural network accelerator is provided, comprising: a partitioning module, configured to determine a plurality of first spatial information corresponding to a neural network model, wherein the first spatial information is used to represent at least one neural network layer in the neural network model;

[0009] The lookup module is used to search for the policy mapping relationship in the cache based on the first spatial information to obtain the first policy information corresponding to at least one first spatial information; the policy mapping relationship includes multiple target spatial information and the target policy information corresponding to each target spatial information, and the multiple target spatial information includes at least one first spatial information;

[0010] The generation module is used to determine the overall optimization strategy corresponding to the neural network model based on the dynamic programming algorithm and multiple policy information, so as to generate instructions that can be executed by the neural network accelerator; wherein, the multiple policy information includes first policy information corresponding to at least one first space information.

[0011] According to one aspect of this disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing any of the methods described above.

[0012] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement any of the methods described above. Attached Figure Description

[0013] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain the application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0014] Figure 1 This is an exemplary structural diagram of the system to which this disclosure applies.

[0015] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an exemplary embodiment of the present disclosure.

[0016] Figure 3 This is a schematic diagram of the structure of another electronic device provided in an exemplary embodiment of the present disclosure.

[0017] Figure 4 This is one of the flowcharts illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0018] Figure 5 This is a schematic diagram of the structure of a neural network model provided in an exemplary embodiment of the present disclosure.

[0019] Figure 6 This is a second schematic flowchart of an instruction generation method for a neural network accelerator provided in an exemplary embodiment of this disclosure.

[0020] Figure 7 This is the third flowchart illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0021] Figure 8 This is the fourth flowchart illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0022] Figure 9 This is the fifth flowchart illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0023] Figure 10 This is a sixth flowchart illustrating an exemplary embodiment of the instruction generation method for a neural network accelerator provided in this disclosure.

[0024] Figure 11 This is the seventh flowchart illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0025] Figure 12 This is the eighth flowchart illustrating an exemplary embodiment of the present disclosure of a method for generating instructions for a neural network accelerator.

[0026] Figure 13 This is a structural diagram of an instruction generation apparatus provided in an exemplary embodiment of the present disclosure.

[0027] Figure 14 This is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. Detailed Implementation

[0028] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0029] Neural network compilers are used to compile and optimize neural network models so that they can run efficiently on hardware devices. Neural network models are mostly described using source code in high-level programming languages ​​(e.g., Java, Python). Neural network compilers can compile and optimize the execution logic of this high-level programming language source code to generate machine code that can run efficiently on hardware devices.

[0030] The neural network compiler optimizes the neural network model by traversing all neural network layers, dividing each layer into multiple partitions (also known as layer spaces), determining the optimization strategy for each partition, and finally obtaining the optimized compilation result of the neural network model based on the optimization strategy for each partition.

[0031] As neural network models grow larger and more complex, the number of neural network layers also increases. Consequently, the number of possible partitions for these layers also increases exponentially. Since the optimization strategy for each partition is calculated, the more partitions there are, the more optimization strategies the neural network compiler needs to determine. This inevitably leads to significant time consumption and negatively impacts the user experience of the neural network compiler.

[0032] In such cases, manual intervention can often reduce the number of partitions. For example, some partitions can be manually pruned, eliminating the need to determine the optimization strategy for each pruned partition and thus reducing the compilation optimization time of the neural network compiler. However, the optimization strategy for manually pruned partitions may be a better fit for the application scenario of the neural network model compared to the optimization strategy for intact partitions. In this case, the compilation optimization result obtained using the optimization strategy for intact partitions will be worse and will not achieve the desired effect.

[0033] Therefore, to address the issue of long compilation optimization time, this disclosure provides an instruction generation method for neural network accelerators. This method first divides the neural network model into multiple first spatial information segments. Then, it searches for policy information corresponding to the multiple first spatial information segments in a policy mapping relationship. Finally, using the policy information and a dynamic programming algorithm, it determines the overall optimization policy corresponding to the neural network model, i.e., the compilation optimization result of the neural network model. Neural network models are typically very large, and optimization policies (i.e., target policy information) corresponding to multiple spaces (i.e., target space information) can be pre-stored using a policy mapping relationship. Thus, when the first spatial information and target space information are the same, the target policy information corresponding to the target space information can be directly used as the first policy information corresponding to the first spatial information, thereby eliminating the process of determining the first policy information and reducing the compilation optimization time of the neural network compiler.

[0034] Figure 1 An instruction generation system is provided for embodiments of this disclosure. The system includes a compilation-side device 11 and a runtime-side device 12. The compilation-side device 11 is used to generate compilation instructions corresponding to a neural network model. The runtime-side device 12 is used to execute the neural network model by running the compilation instructions provided by the compilation-side device 11. These operations include, but are not limited to, convolution operations, pooling operations, activation function operations, etc.

[0035] In some embodiments, the compiler-side device 11 may be implemented by an electronic device 20. The electronic device 20 may include, but is not limited to, a personal computer, a server computer, a multiprocessor system, a microprocessor-based system, etc.

[0036] In some embodiments, the operation-side device 12 may be implemented by an electronic device 30. The electronic device 30 may include, but is not limited to, a neural network accelerator or a neural network chip.

[0037] Figure 2 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present disclosure, as shown below. Figure 2 As shown, the electronic device 20 includes at least one processor 21 and a memory 22.

[0038] The processor 21 is used to execute the instruction generation method for a neural network accelerator in the embodiments of this disclosure. The processor 21 can be a high-performance central processing unit (CPU), graphics processing unit (GPU), or other processor that supports the compilation of neural network instruction sequences, or other processing units with compilation processing capabilities and / or instruction execution capabilities.

[0039] The memory 22 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random-access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 21 may execute the program instructions to implement the instruction generation method for neural network accelerators described below.

[0040] In some embodiments, such as Figure 2 As shown, the electronic device 20 may also include an input device 23 and an output device 24. The input device 23 may include, for example, a keyboard, a mouse, etc., for compilers to input parameters of the neural network, such as instruction sequences of multiple PEs. The output device 24 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc., for compilers to view the final compilation results or intermediate data during the compilation process.

[0041] It should be noted that the specific structure of the electronic device 20 is not limited in the embodiments disclosed herein, and the electronic device 20 may include components such as... Figure 2 The number of components shown may be more or less. For example, electronic device 20 may also include components such as buses and input / output interfaces.

[0042] Figure 3 This is a schematic diagram of the structure of an electronic device 30 provided in an embodiment of this disclosure, as shown below. Figure 3 As shown, the electronic device 30 may include a control unit 31 and a computing unit 32. The control unit 31 is used to run instructions to control the computing unit 32 to execute the instruction generation method of the neural network accelerator described below.

[0043] The control unit 31 may include one or more processors or other processing units with neural network computing capabilities and / or instruction execution capabilities. The processor may include, but is not limited to, GPUs, brain processing units (BPUs), tensor processing units (TPUs), and other processors that support neural network-related computations. In some embodiments, the control unit 31 is used to control the computing unit 32 to perform corresponding operations according to instructions from the neural network accelerator (e.g., convolution operation instructions, pooling operation instructions, and activation function operation instructions).

[0044] The computing unit 32 includes an arithmetic circuit that can be configured to perform neural network operations, including but not limited to any one or more of operations such as convolution, pooling, and activation function operations.

[0045] In some embodiments, the electronic device 30 may further include a buffer memory 33 and an off-chip memory 34. The buffer memory 33 may include one or more independent cache memories or a processing unit with data caching capabilities, and may access the off-chip memory 34 under the control of the control unit 31. In some embodiments, the buffer memory 33 may be a fast, small-capacity static access memory, such as static random access memory (SRAM). Exemplarily, the buffer memory 33 may be configured to receive at least a portion of the input feature data and / or the data of the convolution kernel from the off-chip memory 34 and cache it for use by the computing unit 32, and may also be configured to receive and cache the output feature data output by the computing unit 32, and output the output feature data to the off-chip memory 34.

[0046] The off-chip memory 34 may include one or more independent memories or processing units with data storage capabilities, and may be accessed by the buffer memory 33 under the control of the control unit 31. In some embodiments, the off-chip memory 34 is a slow, large-capacity dynamic access memory, such as dynamic random access memory (DRAM). In some embodiments, the off-chip memory 34 is double data rate (DDR).

[0047] It should be noted that the specific structure of the electronic device 30 is not limited in the embodiments disclosed herein, and the electronic device 30 may include components such as... Figure 3 The number of components shown may be more or less. For example, electronic device 30 may also include components such as buses.

[0048] The methods described in the following embodiments of this disclosure can be applied to many fields that require neural network computation, such as image processing and intelligent driving. For example, image data can be processed in the field of image processing by performing the methods provided in the embodiments of this disclosure.

[0049] Figure 4 This disclosure provides an instruction generation method for neural network accelerators, which can be applied to compiler-side devices, such as... Figure 4 As shown, the method includes steps 401-403.

[0050] Step 401: Determine multiple first-space information corresponding to the neural network model.

[0051] The first spatial information is used to represent at least one neural network layer in the neural network model.

[0052] In some embodiments, the neural network model can be a model applied in fields such as natural language processing, computer vision, speech recognition, and image processing. This application does not limit the application scenarios of the neural network model.

[0053] A neural network model consists of multiple neural network layers. For example, a neural network model may include more than 500 neural network layers. This application does not limit the number of layers in a neural network model; the actual application shall prevail. Figure 5 An exemplary neural network model 50 is shown, which includes neural network layer 1, neural network layer 2, neural network layer 3, neural network layer 4, and neural network layer 5. Dividing the multiple neural network layers in the neural network model yields multiple hierarchical spaces corresponding to the model. Processing these hierarchical spaces separately yields multiple first spatial information. Each of these first spatial information includes data information from the neural network layers.

[0054] The data information for a neural network layer includes the type of operation performed on that layer, the size of the input data, and hyperparameters. The types of operations performed on a neural network layer include, but are not limited to, any one or more of the following: convolution, pooling, and activation function operations.

[0055] The input data for a neural network layer can be image data or text data; this application does not impose any restrictions on this. For example, if the input data for a neural network layer is image data, and the image data size is... Here, H refers to the length of the image, W is the width of the image, and C refers to the number of channels of the image. Generally, grayscale images have 1 channel, and color images have 3 channels.

[0056] The types of computational operations corresponding to different neural network layers can be the same or different. For example, if different neural network layers correspond to different types of computational operations, then the corresponding hyperparameters will also be different. For instance, when the neural network layer is a convolutional layer, the corresponding hyperparameters include the size of the convolutional kernel, the number of convolutional kernels, the stride of the convolution, and the amount of zero padding. As another example, when the neural network layer is a pooling layer, the corresponding hyperparameters include the size of the pooling kernel, the stride of the pooling, and the amount of zero padding.

[0057] In some embodiments, such as Figure 6 As shown above, in the above Figure 4 Based on the illustrated embodiment, step 401 above includes steps 601-603.

[0058] Step 601: Determine the model parameters of the neural network model.

[0059] The model parameters include the number of neural network layers and / or the input / output types of the neural network layers. In some embodiments, the input / output types of the neural network layers include at least one of single-input single-output, single-input double-output, and double-input double-output.

[0060] Most neural network models are described using high-level language source code. This high-level language source code can be converted to obtain the optimization space corresponding to the neural network model. The optimization space provides a more intuitive way to determine the model parameters of the neural network model.

[0061] One method of converting high-level language source code is to input the source code into model visualization software (e.g., Netron), which then outputs the optimization space corresponding to the neural network model. For example, the model visualization software outputs... Figure 5 The diagram shows a directed acyclic graph (DAG). This DAG is an optimization space. A DAG describes a neural network model using a graph structure. In a DAG, the neural network model is abstracted as nodes and directed edges. Nodes represent neural network layers (layer 1, layer 2, layer 3, layer 4, and layer 5), and directed edges represent the input data corresponding to each neural network layer.

[0062] Transforming high-level language source code can also involve analyzing the statements within the source code to extract multiple neural network layers and their corresponding input data from the neural network model. Then, based on the extracted neural network layers and their corresponding input data, a sequence of computational instructions is generated. This sequence of instructions constitutes an optimization space.

[0063] The optimization space provides a more vivid illustration of the model parameters of a neural network model. For example, Figure 5 The optimization space described by the Directed Acyclic Graph (DAG) is shown, illustrating that the neural network model 50 comprises five neural network layers: layer 1, layer 2, layer 3, layer 4, and layer 5. Layer 1 has a single-input, single-output (SIO) input-output type. Layer 2 also has a SIO input-output type. Layer 3 has a SIO input-double-output (DDO) input-output type. Layer 4 has a SIO input-single-output (DDO) input-output type. Layer 5 has a DDO input-double-output (DDO) input-single-output (DDO) input-single-output (DDO) type.

[0064] It should be noted that the input and output types of a neural network layer are mainly related to the neural network layer in the neural network model. The input and output types of a neural network model can also be three-input three-output or four-input three-output, etc. This application does not impose any restrictions on this.

[0065] Step 602: Based on the model parameters of the neural network model, obtain the multiple hierarchical spaces corresponding to the neural network layers in the neural network model.

[0066] Based on the model parameters of the neural network model, the neural network layers in the model are divided into multiple hierarchical spaces. Each hierarchical space includes at least one neural network layer. For example, the division of the neural network layers in the neural network model can be based on preset rules. These preset rules can be related to the number of neural network layers, the input / output types of the neural network layers, or both. This application does not limit the preset rules.

[0067] For example, the preset rule is to divide every two neural network layers, resulting in multiple hierarchical spaces. Combined with... Figure 5 The resulting hierarchical space consists of five layers: layer a (neural network layer 1 - neural network layer 2), layer b (neural network layer 2 - neural network layer 3), layer c (neural network layer 3 - neural network layer 4), layer d (neural network layer 4 - neural network layer 5), and layer e (neural network layer 3 and neural network layer 5).

[0068] Alternatively, the preset rule is to divide the neural network into multiple hierarchical spaces by dividing it into every three neural network layers. Combined with... Figure 5 The resulting hierarchical space consists of four layers: layer f (neural network layer 1 to neural network layer 3), layer g (neural network layer 2 to neural network layer 4), layer h (neural network layer 3 to neural network layer 5), and layer i (neural network layer 2, neural network layer 3, and neural network layer 5).

[0069] Alternatively, the preset rule is to divide the neural network according to the input and output types of the layers, resulting in multiple hierarchical spaces. Combined with... Figure 5The resulting hierarchical spaces are four: hierarchical space h (neural network layers 1-2), hierarchical space l (neural network layer 3), hierarchical space j (neural network layer 4), and hierarchical space k (neural network layer 5). Among them, neural network layers 1 and 2 have a single-input single-output input-output type, neural network layer 3 has a single-input double-output type, neural network layer 4 has a single-input single-output type, and neural network layer 5 has a double-input single-output type. Although the input-output type of neural network layer 4 is the same as that of neural network layers 1 and 2, the input data of neural network layer 4 depends on the output data of neural network layer 3. Therefore, neural network layer 4 cannot be classified into hierarchical space 1 and needs to be separately assigned to hierarchical space 3.

[0070] by Figure 5 Taking the neural network model 50 shown as an example, the neural network model 50 can be divided into multiple hierarchical spaces, and the details of the multiple hierarchical spaces are shown in Table 1 below.

[0071] Table 1

[0072]

[0073] As shown in Table 1, the neural network model 50 can be divided into 18 hierarchical spaces. These are hierarchical space 1, hierarchical space 2, hierarchical space 3, hierarchical space 4, hierarchical space 5, hierarchical space 6, hierarchical space 7, hierarchical space 8, hierarchical space 9, hierarchical space 10, hierarchical space 11, hierarchical space 12, hierarchical space 13, hierarchical space 14, hierarchical space 15, hierarchical space 16, hierarchical space 17, and hierarchical space 18.

[0074] The layered space is as follows: Layer 1 consists of neural network layer 1. Layer 2 consists of neural network layer 2. Layer 3 consists of neural network layer 3. Layer 4 consists of neural network layer 4. Layer 5 consists of neural network layer 5. Layer 6 consists of neural network layers 1-2 (specifically including neural network layer 1 and neural network layer 2). Layer 7 consists of neural network layers 2-3 (specifically including neural network layer 2 and neural network layer 3). Layer 8 consists of neural network layers 3-4 (specifically including neural network layer 3 and neural network layer 4). Layer 9 consists of neural network layers 4-5 (specifically including neural network layer 4 and neural network layer 5). Layered space 10 consists of neural network layers 3 and 5 (specifically, neural network layers 3 and 5). Layered space 11 consists of neural network layers 1-3 (specifically, neural network layers 1, 2, and 3). Layered space 12 consists of neural network layers 2-4 (specifically, neural network layers 2, 3, and 4). Layered space 13 consists of neural network layers 3-5 (specifically, neural network layers 3, 4, and 5). Layered space 14 consists of neural network layers 2-3 and 5 (specifically, neural network layers 2, 3, and 5). Layer 15 consists of neural network layers 1-4 (specifically, neural network layers 1, 2, 3, and 4). Layer 16 consists of neural network layers 2-5 (specifically, neural network layers 2, 3, 4, and 5). Layer 17 consists of neural network layers 1-3 and 5 (specifically, neural network layers 1, 2, 3, and 5). Layer 18 consists of neural network layers 1-5 (specifically, neural network layers 1, 2, 3, 4, and 5).

[0075] Step 603: Based on multiple hierarchical spaces, obtain multiple first-space information.

[0076] In some embodiments, obtaining multiple first spatial information based on multiple hierarchical spaces specifically involves encoding multiple hierarchical spaces to obtain multiple first spatial information, with each hierarchical space corresponding one-to-one with the multiple first spatial information.

[0077] In some embodiments, such as Figure 7 As shown above, in the above Figure 6 Based on the illustrated embodiment, step 603 above includes step 701.

[0078] Step 701: Encode the hardware resource information of multiple hierarchical spaces and neural network accelerators to obtain multiple first space information.

[0079] Since the neural network model compiled and optimized by the neural network encoder is executed on a hardware device, after obtaining the multiple layer spaces corresponding to the neural network model, it is also necessary to consider the hardware device on which the neural network model runs. Therefore, once the hardware device on which the neural network model runs is determined, the multiple layer spaces corresponding to the neural network model and the hardware resource information of the hardware device are combined together for encoding. For example, if it is determined that the neural network model runs on a neural network accelerator, then each layer space in the multiple layer spaces and the hardware resource information of the neural network accelerator are encoded separately.

[0080] In some embodiments, encoding the hardware resource information of multiple hierarchical spaces and neural network accelerators to obtain first space information can be achieved through a hash algorithm. The encoded first space information can be represented by multiple key values. Each key value corresponds one-to-one with a different hierarchical space, uniquely identifying the multiple hierarchical spaces and neural network accelerators.

[0081] Furthermore, multiple mapping relationships can exist during the encoding process, and the processor stores these multiple mapping relationships. These mapping relationships indicate a first spatial information, as well as the corresponding hierarchical space and the hardware resource information of the neural network accelerator. The hierarchical spaces stored in multiple mapping relationships are different. The first spatial information stored in multiple mapping relationships is also different.

[0082] Encoding the hardware resource information of each hierarchical space and the neural network accelerator within the multiple hierarchical spaces can be achieved using other algorithms, as long as a unique identifier representing the hierarchical space and the neural network accelerator is obtained. This application does not restrict the encoding algorithm for the hierarchical spaces and the neural network accelerator. The encoded first spatial information occupies less storage space compared to the unencoded hierarchical spaces and the neural network accelerator's hardware resource information. Furthermore, subsequent data processing using the multiple first spatial information requires less data resources and less hardware resources compared to data processing using the unencoded hierarchical spaces and the neural network accelerator's hardware resource information. Therefore, this encoding method can reduce hardware resource consumption to a certain extent.

[0083] Step 402: Based on the first spatial information, search for the policy mapping relationship in the cache to obtain the first policy information corresponding to at least one piece of first spatial information.

[0084] The strategy mapping relationship includes multiple target space information and target strategy information corresponding to each target space information. The multiple target space information includes at least one first space information.

[0085] As the scale of the neural network model increases (i.e., the number of neural network layers increases), as seen in steps 601-701, the number of layer spaces corresponding to the neural network model also increases, and the number of first-space information corresponding to each layer space also increases. Then, the first policy information corresponding to each first-space information is determined, and finally, based on multiple first-policy information, the overall optimization policy corresponding to the neural network model is derived. Since the first policy information of multiple first-space information can only be obtained through computation, the more first-space information there is, the more first-policy information corresponding to the first-space information needs to be determined. Therefore, the computational load will increase significantly, and the computation time will also be longer.

[0086] To address this, this application utilizes a policy mapping relationship to reduce the process of determining the first policy information corresponding to the first spatial information, thereby reducing time and hardware resource consumption. For example, the policy mapping relationship pre-stores a large amount of target spatial information and the target policy information corresponding to each target spatial information. The target spatial information is stored as a key, and the target policy information corresponding to the target spatial information is stored as a value. After determining multiple first spatial information corresponding to the neural network model, since these multiple first spatial information are also recorded as key values, the key values ​​corresponding to the multiple first spatial information can be directly compared with the key values ​​corresponding to the multiple target spatial information in the policy mapping relationship. Through comparison, the target spatial information that matches the first spatial information is determined from among the multiple target spatial information, and then the target policy information corresponding to the target spatial information is used as the first policy information corresponding to the first spatial information. Thus, for first spatial information that matches the target spatial information, it is not necessary to determine the first policy information corresponding to the first spatial information again; the first policy information can be used directly, thereby reducing compilation optimization time.

[0087] Step 403: Based on the dynamic programming algorithm and multiple policy information, determine the overall optimization policy corresponding to the neural network model to generate instructions that can be executed by the neural network accelerator.

[0088] Among them, the multiple strategy information includes at least one first strategy information corresponding to each of the first spatial information.

[0089] Dynamic programming algorithms analyze multiple policy information to determine the policy with a higher degree of matching with the scenario in which the neural network model is applied. This policy is then used as the overall optimization policy for the neural network model. Finally, based on this overall optimization policy, executable instructions for the neural network accelerator are generated.

[0090] For example, scenarios where neural network models are used may require them to output data in a short time. In such cases, dynamic programming algorithms analyze multiple policy options and select the policy with the shortest execution time as the overall optimal policy. Similarly, scenarios where neural network models are used may require them to consume as little memory bandwidth as possible within the same time frame. In these cases, dynamic programming algorithms analyze multiple policy options and select the policy with the least memory bandwidth consumption as the overall optimal policy.

[0091] Based on this scheme, by pre-setting the policy mapping relationship, it is equivalent to setting up a database in advance, which caches the optimization policies (i.e., target policy information) corresponding to multiple spaces (i.e., target space information). When it is necessary to compile and optimize a new neural network model, the neural network model is first split into multiple spaces (i.e., first space information), and then the optimization policies corresponding to multiple spaces are searched from the database (i.e., policy mapping relationship). If they can be found, it is not necessary to calculate the optimization policy corresponding to the space, thus directly reducing the compilation and optimization time of the neural network compiler.

[0092] In some embodiments, such as Figure 8 As shown above, in the above Figure 4 Based on the illustrated embodiment, the method further includes:

[0093] Step 801: If there is at least one second spatial information that is different from the target spatial information among multiple first spatial information, determine the second strategy information corresponding to each of the at least one second spatial information.

[0094] Among them, the multiple strategy information includes at least one second strategy information corresponding to the second spatial information.

[0095] In the strategy mapping relationship, search for target space information that matches multiple first space information. If the target space information corresponding to multiple first space information is found, then the target strategy information corresponding to each of the found target space information is used as the first strategy information corresponding to each of the multiple first space information. Therefore, the multiple strategy information in step 403 is the first strategy information corresponding to each of the multiple first space information.

[0096] If only some of the target space information is found among multiple first space information, then the target strategy information corresponding to each of the found target space information will be used as the first strategy information corresponding to the partial first space information.

[0097] First spatial information for which no target spatial information was found is identified as second spatial information (i.e., at least one second spatial information that differs from the target spatial information among multiple first spatial information). Then, second strategy information corresponding to each second spatial information is determined sequentially. That is, multiple strategy information includes first strategy information corresponding to some first spatial information and multiple second strategy information corresponding to multiple second spatial information.

[0098] In some embodiments, such as Figure 9 As shown above, in the above Figure 4 Based on the embodiment shown, the "determining the second strategy information corresponding to at least one second spatial information" in step 801 above, "in the case that there is at least one second spatial information different from the target spatial information among multiple first spatial information," includes steps 901-903.

[0099] Step 901: Based on the second spatial information, determine multiple candidate strategies corresponding to the second spatial information.

[0100] The first spatial information is obtained by encoding the hierarchical space, so at least one second spatial information among the multiple first spatial information is also obtained by encoding. As can be seen from step 701, the processor stores multiple mapping relationships during the encoding process. Since the second spatial information is obtained by encoding, the corresponding second space can be found among multiple mapping relationships. The second space includes at least one neural network layer and the data information of the neural network layer. The second space is a part of multiple hierarchical spaces.

[0101] For details on the data information of the neural network layer, please refer to the description in step 401, which will not be repeated here.

[0102] Determining multiple candidate strategies corresponding to the second space information specifically refers to determining multiple candidate strategies corresponding to the neural network layers in the second space. The determination process involves first determining the data size of the input data for the neural network layers in the second space based on the data information. Then, based on the input data size and the hardware resources of the neural network accelerator, multiple candidate strategies are derived.

[0103] Step 902: Determine multiple cost parameters for multiple candidate strategies corresponding to the second spatial information executed by multiple computing nodes.

[0104] Since one piece of second spatial information can correspond to multiple candidate policies, in order to more quickly determine the multiple cost parameters of multiple candidate policies corresponding to at least one piece of second spatial information, at least one piece of second spatial information can be distributed across multiple computing nodes. The computing nodes are then used to determine the multiple cost parameters of the multiple candidate policies corresponding to the second spatial information. The allocation rule for distributing second spatial information to computing nodes is: the computing resources required to process the multiple cost parameters of the multiple candidate policies corresponding to the second spatial information are less than the currently remaining computing resources of the computing node.

[0105] Therefore, a neural network compiler can first obtain the load capacity, data to be processed, and remaining computing resources of each computing node in the processor (e.g., CPU). Then, it assesses the computing resources required to process the second spatial information, and compares the remaining computing resources of each computing node with the computing resources required to process the second spatial information, thereby determining the computing node to process the second spatial information. In the process of determining the computing node, load balancing among the computing nodes can also be considered to avoid overloading a single computing node.

[0106] By running multiple candidate policies for each second space on multiple computing nodes, multiple cost parameters are obtained for each candidate policy. This results in faster processing speed and shorter processing time. These cost parameters include processing time, computational resources consumed, storage resources used, and processing cycle, among others.

[0107] Step 903: Based on multiple cost parameters, determine the second strategy information corresponding to the second spatial information among multiple candidate strategies.

[0108] Based on the application scenario of the neural network model and multiple cost parameters, the second policy information corresponding to the second spatial information is determined from multiple candidate policies. For example, the application scenario of the neural network model is real-time car navigation. The multiple candidate policies corresponding to the second spatial information include candidate policy 1 and candidate policy 2. According to the cost parameters of candidate policy 1, it takes longer to execute the calculation of the neural network layer corresponding to the second spatial information, but consumes less bandwidth. According to the cost parameters of candidate policy 2, it takes less time to execute the calculation of the neural network layer corresponding to the second spatial information, but consumes more bandwidth. To ensure that navigation results can be received in real time while driving, candidate policy 2 is selected as the second policy information in this scenario.

[0109] In some embodiments, such as Figure 10 As shown above, in the above Figure 9 Based on the embodiment shown, step 903 above includes steps 1001-1002.

[0110] Step 1001: Determine the second optimization strategy from multiple candidate strategies based on multiple cost parameters.

[0111] Step 1002: Encode the second optimization strategy to obtain the second strategy information corresponding to the second spatial information.

[0112] Combining step 903, based on the application scenario of the neural network model and multiple cost parameters, a second optimal strategy can be determined from multiple candidate strategies. Then, the second optimal strategy is encoded to obtain second strategy information. This second strategy information can be represented by a value. Since the second optimal strategy involves a large amount of data, encoding it reduces the amount of data required. This not only reduces storage pressure but also computational pressure.

[0113] In some embodiments, such as Figure 11 As shown above, in the above Figure 10 Based on the illustrated embodiment, the above method further includes step 1101.

[0114] Step 1101: In the strategy mapping relationship, add the second strategy information corresponding to the second spatial information.

[0115] As can be seen from step 801, among the multiple first spatial information, there may be at least one second spatial information that differs from the target spatial information. Therefore, after determining the second policy information corresponding to the second spatial information through the computation node, the second spatial information and its corresponding second policy information can be updated in the policy mapping relationship. This expands the data volume of the policy mapping relationship. During the next compilation optimization, the search can be performed in the updated policy mapping relationship, thereby increasing the search range, improving the probability of finding the first policy information, and further reducing the compilation optimization time.

[0116] In some embodiments, such as Figure 12 As shown above, in the above Figure 11 Based on the illustrated embodiment, step 403 above includes steps 1201-1203.

[0117] Step 1201: Based on the first strategy information and the second strategy information, determine the first optimization strategy corresponding to the first strategy information and the second optimization strategy corresponding to the second strategy information.

[0118] Since the first and second policy information are generated after encoding, the processor stores multiple mapping relationships containing both the first and second policy information. When it is necessary to determine the overall optimization policy corresponding to the neural network model based on the first and second policy information, the processor can first look up the first optimization policy corresponding to the first policy information and the second optimization policy corresponding to the second policy information based on the multiple mapping relationships.

[0119] Step 1202: Based on the dynamic programming algorithm, determine the overall optimization strategy corresponding to the neural network model from at least one first optimization strategy and at least one second optimization strategy.

[0120] For example, with Figure 5 Taking the neural network model 50 shown as an example, as shown in Table 1, the neural network model 50 corresponds to 18 hierarchical spaces. Searching the policy mapping relationship reveals that it only includes multiple hierarchical spaces corresponding to neural network layers 1-3, and the first optimization policies corresponding to each of these hierarchical spaces. The multiple hierarchical spaces are hierarchical space 1, hierarchical space 2, hierarchical space 3, hierarchical space 6, hierarchical space 7, and hierarchical space 11. The first optimization policies corresponding to each hierarchical space are the first optimization policy corresponding to hierarchical space 1, hierarchical space 2, hierarchical space 3, hierarchical space 6, hierarchical space 7, and hierarchical space 11.

[0121] Since the policy mapping relationship is determined to contain only multiple hierarchical spaces corresponding to neural network layers 1 to 3 and the first optimization strategies corresponding to each of the multiple hierarchical spaces, the multiple hierarchical spaces related to neural network layer 4 or neural network layer 5 and the second optimization strategies corresponding to each of the multiple hierarchical spaces need to be re-determined.

[0122] For example, using steps 901-1002, multiple layer spaces associated with neural network layer 4 or neural network layer 5, and the second optimization strategies corresponding to each of the multiple layer spaces, can be determined. The multiple layer spaces associated with neural network layer 4 or neural network layer 5 are layer space 4, layer space 5, layer space 8, layer space 9, layer space 10, layer space 12, layer space 13, layer space 14, layer space 15, layer space 16, layer space 17, and layer space 18. The second optimization strategies corresponding to the multiple layer spaces associated with neural network layer 4 or neural network layer 5 are the second optimization strategies corresponding to layer space 4, layer space 5, layer space 8, layer space 9, layer space 10, layer space 12, layer space 13, layer space 14, layer space 15, layer space 16, layer space 17, and layer space 18.

[0123] Based on the dynamic programming algorithm, according to the first optimization strategy corresponding to layer space 1, layer space 2, layer space 3, layer space 6, layer space 7, layer space 11, layer space 4, layer space 5, layer space 8, layer space 9, layer space 10, layer space 12, layer space 13, layer space 14, layer space 15, layer space 16, layer space 17, and layer space 18, the overall optimization strategy for the neural network model 50 is determined to be the first optimization strategy corresponding to layer space 11, the second optimization strategy corresponding to layer space 4, and the second optimization strategy corresponding to layer space 5.

[0124] Step 1203: Based on the overall optimization strategy corresponding to the neural network model, generate executable instructions for the neural network accelerator.

[0125] In this embodiment, when generating executable instructions for the neural network accelerator, the compilation-side device first divides the neural network model into multiple first spatial information. Then, it searches for multiple first policy information corresponding to the multiple first spatial information in the policy mapping relationship. If not all of the multiple first spatial information are found in the policy mapping relationship, it determines multiple second policy information corresponding to the unfound spatial information (i.e., second spatial information). Finally, using the multiple policy information (i.e., first policy information and second policy information) and a dynamic programming algorithm, it determines the overall optimization policy corresponding to the neural network model, i.e., the compilation optimization result of the neural network model. Neural network models are typically very large, and optimization policies (i.e., target policy information) corresponding to some spaces (i.e., target spatial information) can be pre-stored using the policy mapping relationship. Thus, when the first spatial information and target spatial information are the same, the target policy information corresponding to the target spatial information can be directly used as the first policy information corresponding to the first spatial information, thereby eliminating the process of determining the first policy information and reducing the compilation optimization time of the neural network compiler.

[0126] Figure 13 An instruction generation apparatus provided in this disclosure embodiment, such as Figure 13 As shown, the instruction generation device 1300 includes a partitioning module 1301, a search module 1302, and a generation module 1303.

[0127] The partitioning module 1301 is used to determine multiple first spatial information corresponding to the neural network model, and the first spatial information is used to represent at least one neural network layer in the neural network model.

[0128] The lookup module 1302 is used to look up the policy mapping relationship in the cache according to the first spatial information to obtain the first policy information corresponding to at least one first spatial information; the policy mapping relationship includes multiple target spatial information and the target policy information corresponding to each target spatial information, and the multiple target spatial information includes at least one first spatial information;

[0129] The generation module 1303 is used to determine the overall optimization strategy corresponding to the neural network model based on the dynamic programming algorithm and multiple strategy information, so as to generate instructions executable by the neural network accelerator; wherein, the multiple strategy information includes first strategy information corresponding to at least one first space information respectively.

[0130] In some embodiments, the instruction generation apparatus 1300 further includes a determination module. The determination module is configured to determine second strategy information corresponding to each of the at least one second spatial information if at least one second spatial information differs from the target spatial information among the plurality of first spatial information; the plurality of strategy information includes the second strategy information corresponding to each of the at least one second spatial information.

[0131] In some embodiments, the determining module includes: a first determining unit, a second determining unit, and a third determining unit. The first determining unit is configured to determine multiple candidate strategies corresponding to the second spatial information based on the second spatial information; the second determining unit is configured to determine multiple cost parameters for multiple computing nodes to execute the multiple candidate strategies corresponding to the second spatial information; and the third determining unit is configured to determine second strategy information corresponding to the second spatial information among the multiple candidate strategies based on the multiple cost parameters.

[0132] In some embodiments, the third determining unit is further configured to determine a second optimization strategy among multiple candidate strategies based on multiple cost parameters; and to encode the second optimization strategy to obtain second strategy information corresponding to the second spatial information.

[0133] In some embodiments, the instruction generation apparatus 1300 includes an addition module, which is used to add second strategy information corresponding to the second spatial information in the strategy mapping relationship.

[0134] In some embodiments, the partitioning module 1301 is further configured to determine the model parameters of the neural network model; the model parameters include the number of neural network layers and / or the input and output types of the neural network layers; based on the model parameters of the neural network model, multiple hierarchical spaces corresponding to the neural network layers in the neural network model are obtained; based on the multiple hierarchical spaces, multiple first space information is obtained.

[0135] In some embodiments, the input-output type of the neural network layer includes at least one of single-input single-output, single-input double-output, and double-input double-output, and the model parameters are represented by a directed acyclic graph or by a sequence of computation instructions.

[0136] In some embodiments, the partitioning module 1301 is further configured to encode the hardware resource information of multiple hierarchical spaces and neural network accelerators to obtain first spatial information.

[0137] In some embodiments, the generation module 1303 is further configured to determine a first optimization strategy corresponding to the first strategy information and a second optimization strategy corresponding to the second strategy information based on the first strategy information and the second strategy information; determine an optimization strategy corresponding to the neural network model among at least one first optimization strategy and at least one second optimization strategy based on a dynamic programming algorithm; and generate instructions executable by the neural network accelerator based on the optimization strategy corresponding to the neural network model.

[0138] Figure 14 A structural diagram of an electronic device provided in this disclosure embodiment, such as... Figure 14 As shown, the electronic device 1400 includes one or more processors 1401 and memory 1402.

[0139] The processor 1401 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1400 to perform desired functions.

[0140] The memory 1402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1401 may execute the program instructions to implement the instruction generation methods of the various embodiments of this disclosure described above and / or other desired functions.

[0141] In one example, the electronic device 1400 may also include an input device 1403 and an output device 1404, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0142] Of course, for the sake of simplicity, Figure 14Only some of the components of the electronic device 1400 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1400 may include any other suitable components depending on the specific application.

[0143] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the instruction generation methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0144] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0145] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the instruction generation methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0146] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0147] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0148] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0149] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0150] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0151] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for generating instructions for a neural network accelerator, comprising: Determine multiple first spatial information corresponding to the neural network model. The first spatial information is obtained from the hierarchical space corresponding to the neural network layer in the neural network model and is used to represent at least one neural network layer in the neural network model. Based on the first spatial information, a policy mapping relationship is searched in the cache to obtain first policy information corresponding to at least one first spatial information; the policy mapping relationship includes multiple target spatial information and target policy information corresponding to each of the target spatial information, and the multiple target spatial information includes the at least one first spatial information; Based on a dynamic programming algorithm and multiple policy information, a total optimization policy corresponding to the neural network model is determined to generate executable instructions for the neural network accelerator; wherein, the multiple policy information includes first policy information corresponding to at least one first spatial information.

2. The method according to claim 1, further comprising: If at least one second spatial information that is different from the target spatial information exists among the plurality of first spatial information, second strategy information corresponding to the at least one second spatial information is determined; the plurality of strategy information includes the second strategy information corresponding to the at least one second spatial information.

3. The method according to claim 2, wherein, The step of determining the second strategy information corresponding to each of the at least one second spatial information includes: Based on the second spatial information, multiple candidate strategies corresponding to the second spatial information are determined; Determine multiple cost parameters for multiple candidate strategies corresponding to the second spatial information executed by multiple computing nodes; Based on the plurality of cost parameters, the second strategy information corresponding to the second spatial information is determined from the plurality of candidate strategies.

4. The method according to claim 3, wherein, Based on the plurality of cost parameters, determining the second policy information corresponding to the second spatial information from the plurality of candidate policies includes: Based on the plurality of cost parameters, a second optimization strategy is determined from the plurality of candidate strategies; The second optimization strategy is encoded to obtain the second strategy information corresponding to the second spatial information.

5. The method according to any one of claims 2-4, wherein, The method further includes: In the strategy mapping relationship, the second strategy information corresponding to the second spatial information is added.

6. The method according to any one of claims 1-4, wherein, The determination of multiple first spatial information corresponding to the neural network model includes: Determine the model parameters of the neural network model; the model parameters include the number of neural network layers, and / or the input and output types of the neural network layers; Based on the model parameters of the neural network model, multiple hierarchical spaces corresponding to the neural network layers in the neural network model are obtained; Based on the multiple hierarchical spaces, multiple first spatial information is obtained.

7. The method according to claim 6, wherein, The input-output type of the neural network layer includes at least one of single-input single-output, single-input double-output, and double-input double-output. The model parameters are represented by a directed acyclic graph or by a sequence of computation instructions.

8. The method according to claim 6, wherein, The process of obtaining multiple first spatial information based on the multiple hierarchical spaces includes: The hardware resource information of the multiple hierarchical spaces and the neural network accelerator are encoded to obtain multiple first space information.

9. The method according to any one of claims 2-4, wherein, The process of determining the overall optimization strategy corresponding to the neural network model based on dynamic programming algorithm and multiple strategy information to generate executable instructions for the neural network accelerator includes: Based on the first strategy information and the second strategy information, a first optimization strategy corresponding to the first strategy information and a second optimization strategy corresponding to the second strategy information are determined. Based on the dynamic programming algorithm, the overall optimization strategy corresponding to the neural network model is determined among at least one first optimization strategy and at least one second optimization strategy; Based on the overall optimization strategy corresponding to the neural network model, executable instructions for the neural network accelerator are generated.

10. An instruction generation apparatus for a neural network accelerator, comprising: A partitioning module is used to determine multiple first spatial information corresponding to the neural network model. The first spatial information is obtained from the hierarchical space corresponding to the neural network layer in the neural network model and is used to represent at least one neural network layer in the neural network model. The lookup module is used to look up a policy mapping relationship in the cache based on the first spatial information to obtain at least one first policy information corresponding to the first spatial information; the policy mapping relationship includes multiple target spatial information and target policy information corresponding to each of the target spatial information, and the multiple target spatial information includes the at least one first spatial information; A generation module is used to determine the overall optimization strategy corresponding to the neural network model based on a dynamic programming algorithm and multiple strategy information, so as to generate instructions executable by the neural network accelerator; wherein, the multiple strategy information includes first strategy information corresponding to the at least one first spatial information.

11. A computer-readable storage medium storing a computer program for performing the instruction generation method for a neural network accelerator as described in any one of claims 1-9.

12. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the instruction generation method for a neural network accelerator as described in any one of claims 1-9.

Images