Method and apparatus for compiling a neural network model to obtain executable instructions

Abstract

A method and apparatus for compiling a neural network model to obtain executable instructions are disclosed. The method includes: determining a first data block storage mode supported by a first operator and a second data block storage mode supported by a second operator in the neural network model; determining target conversion mode information for converting the first data block storage mode to the second data block storage mode based on a conversion operation supported by a neural network accelerator for data block storage mode conversion; generating a conversion operator whose input is the output of the first operator, whose output is the input of the second operator, and which is used to perform data block storage mode conversion according to the target conversion mode information; and generating executable instructions corresponding to the neural network model through compilation processing based on the operators and the conversion operator in the neural network model. This disclosed embodiment can guarantee the correct operation of the operators and ensure that the neural network model runs normally and reliably on the neural network accelerator.

Description

Technical Field

[0001] This disclosure relates to artificial intelligence (AI) technology, and in particular to a method and apparatus for compiling a neural network model to obtain executable instructions. Background Technology

[0002] In the field of artificial intelligence, neural network models are widely used. However, in neural network models, there may be mismatches in the data grouping and storage methods supported by adjacent operators, which can easily lead to errors in the operation of operators that are listed later in the sequence. How to effectively address this situation to ensure the correct operation of operators is a problem worthy of attention for those skilled in the art. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides a method and apparatus for compiling a neural network model to obtain executable instructions, thereby ensuring the correct operation of the operator and guaranteeing the normal and reliable operation of the neural network model on a neural network accelerator.

[0004] According to one aspect of this disclosure, a method is provided for compiling a neural network model to obtain executable instructions, comprising:

[0005] Determine the first data grouping storage method supported by the first operator and the second data grouping storage method supported by the second operator in the neural network model; wherein, the first operator is the preceding operator of the second operator;

[0006] Based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, the target conversion mode information for converting the first data packet storage mode to the second data packet storage mode is determined;

[0007] A conversion operator is generated whose input is the output of the first operator, whose output is the input of the second operator, and is used to convert the data grouping storage method according to the target conversion method information;

[0008] Based on the operators and transformation operators in the neural network model, executable instructions corresponding to the neural network model are generated through compilation.

[0009] According to another aspect of this disclosure, an apparatus for compiling a neural network model to obtain executable instructions is provided, comprising:

[0010] The first determining module is used to determine the first data grouping storage method supported by the first operator and the second data grouping storage method supported by the second operator in the neural network model; wherein, the first operator is the preceding operator of the second operator;

[0011] The second determining module is used to determine, based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, the target conversion mode information for converting the first data packet storage mode determined by the first determining module into the second data packet storage mode determined by the first determining module.

[0012] The first generation module is used to generate a conversion operator whose input is the output of the first operator, whose output is the input of the second operator, and whose conversion operator is used to perform data grouping storage method conversion according to the target conversion method information determined by the second determining module;

[0013] The second generation module is used to generate executable instructions corresponding to the neural network model through compilation processing based on the operators in the neural network model and the transformation operators generated by the first generation module.

[0014] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for executing the method described above for compiling a neural network model to obtain executable instructions.

[0015] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0016] processor;

[0017] Memory used to store the processor's executable instructions;

[0018] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method described above for compiling a neural network model to obtain executable instructions.

[0019] According to another aspect of the present disclosure, a computer program product is provided, which, when the instructions in the computer program product are executed by a processor, performs the above-described method for compiling a neural network model to obtain executable instructions.

[0020] Based on the methods, apparatus, storage media, electronic devices, and products for compiling neural network models to obtain executable instructions provided in the above embodiments of this disclosure, the neural network model may involve two stages: a compilation stage and a runtime stage. In the compilation stage, for any two adjacent operators in the neural network model, such as a first operator and a second operator, if the first data block storage method supported by the first operator and the second data block storage method supported by the second operator are different, a target conversion method information for converting the first data block storage method to the second data block storage method can be determined based on the conversion operation for data block storage method conversion supported by the neural network accelerator. A conversion operator is then generated, with the first operator's output as input, the second operator's input as output, and used to perform data block storage method conversion according to the target conversion method information. This conversion operator, along with all operators in the neural network model, is then used for the compilation processing of the neural network model. During the runtime phase, the executable instructions generated during the compilation phase are executed by the neural network accelerator, which is equivalent to running the neural network model on the neural network accelerator. In this process, after the output of the first operator is stored in the on-chip memory according to the first data grouping storage method, the conversion operator can convert the data grouping storage method of the output of the first operator in the on-chip memory according to the target conversion method information. The conversion result can be used as the input of the second operator. This ensures that the input of the second operator conforms to the second data grouping storage method supported by the second operator, thereby helping to ensure the correct operation of the second operator and thus ensuring the normal and reliable operation of the neural network model on the neural network accelerator. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a method for compiling a neural network model to obtain executable instructions, provided by some exemplary embodiments of this disclosure.

[0022] Figure 2 This is a schematic diagram of the storage hierarchy of the on-chip memory in some exemplary embodiments of this disclosure.

[0023] Figure 3-1 This is one of the schematic diagrams of data group storage methods in some exemplary embodiments of this disclosure.

[0024] Figure 3-2 This is a second schematic diagram of a data group storage method in some exemplary embodiments of this disclosure.

[0025] Figure 4-1 This is one of the schematic diagrams of data packet storage methods in some other exemplary embodiments of this disclosure.

[0026] Figure 4-2 This is a second schematic diagram of a data packet storage method in some other exemplary embodiments of this disclosure.

[0027] Figure 5-1 This is one of the schematic diagrams of a data group storage method in some exemplary embodiments of this disclosure.

[0028] Figure 5-2 This is a second schematic diagram of a data packet storage method in some exemplary embodiments of this disclosure.

[0029] Figure 6 This is a flowchart illustrating a method for determining target conversion information provided by some exemplary embodiments of this disclosure.

[0030] Figure 7 This is a flowchart illustrating a method for determining target conversion information provided by other exemplary embodiments of this disclosure.

[0031] Figure 8 This is a flowchart illustrating a method for generating transformation operators provided by some exemplary embodiments of this disclosure.

[0032] Figure 9-1 This is one of the schematic diagrams of a data group storage method in some exemplary embodiments of this disclosure.

[0033] Figure 9-2 This is a second schematic diagram of a data group storage method in some exemplary embodiments of this disclosure.

[0034] Figure 9-3 This is the third of several exemplary embodiments of the present disclosure, illustrating a data grouping storage method.

[0035] Figure 9-4 This is the fourth schematic diagram of a data group storage method in some exemplary embodiments of this disclosure.

[0036] Figure 9-5 This is the fifth of several exemplary embodiments of the present disclosure, illustrating a data grouping storage method.

[0037] Figure 9-6 This is the sixth of several exemplary embodiments of the present disclosure illustrating a data grouping storage method.

[0038] Figure 10 This is a schematic diagram of the data flow from the first operator to the second operator during the computation phase in some exemplary embodiments of this disclosure.

[0039] Figure 11 This is a schematic diagram of the structure of an apparatus for compiling a neural network model to obtain executable instructions, provided by some exemplary embodiments of the present disclosure.

[0040] Figure 12 This is a schematic diagram of the structure of the second determining module in some exemplary embodiments of this disclosure.

[0041] Figure 13 This is a schematic diagram of the structure of the second determining module in some other exemplary embodiments of this disclosure.

[0042] Figure 14 This is a schematic diagram of the structure of the first generation module in some exemplary embodiments of this disclosure.

[0043] Figure 15 This is a structural diagram of an electronic device provided by some exemplary embodiments of this disclosure. Detailed Implementation

[0044] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0045] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0046] Application Overview

[0047] In the field of artificial intelligence, neural network models have a wide range of applications. For example, neural network models can be used for target detection, target tracking, and target trajectory prediction in autonomous driving scenarios. A neural network model can include multiple operators arranged in a certain order. These operators can include, but are not limited to, convolution operators, pooling operators, rectified linear unit (ReLU) operators, and deconvolution operators. Each operator can have its own corresponding operation type. For example, the operation type corresponding to a convolution operator can be convolution, and the operation type corresponding to a pooling operator can be pooling.

[0048] To improve the parallelism of operators, researchers may specify data grouping and storage methods for some operators in a neural network model. Data grouping and storage methods characterize how data is grouped and stored in on-chip memory. On-chip memory is an important component of a chip. On-chip memory can include Static Random-Access Memory (SRAM).

[0049] Generally, any operator in a neural network model has inputs and outputs. For the inputs and outputs of the same operator, developers can specify different data grouping and storage methods. Optionally, the data grouping and storage methods corresponding to the inputs and outputs of the same operator can be the same or different.

[0050] In developing this disclosure, the inventors discovered that in some cases, for two adjacent operators in a neural network model, such as operator A and operator B (operator A being the preceding operator B), the data grouping storage method corresponding to the output of operator A does not match (i.e., they are different) with the data grouping storage method corresponding to the input of operator B, which can easily lead to errors in the operation of operator B. Therefore, how to effectively address this situation to ensure the correct operation of operator B is a problem worthy of attention for those skilled in the art.

[0051] Exemplary methods

[0052] Figure 1 This is a flowchart illustrating a method for compiling a neural network model to obtain executable instructions, provided by some exemplary embodiments of this disclosure. Figure 1 The method shown can be applied to compilers. Figure 1 The method shown may include steps 110, 120, 130 and 140.

[0053] Step 110: Determine the first data grouping storage method supported by the first operator and the second data grouping storage method supported by the second operator in the neural network model; wherein, the first operator is the operator preceding the second operator.

[0054] It should be noted that the first operator can be any of the multiple operators included in the neural network model, and the second operator can be an operator adjacent to the first operator and ordered after the first operator. For example, the first operator can be operator A mentioned above, and the second operator can be operator B mentioned above.

[0055] Optionally, the first data group storage method supported by the first operator may refer to the data group storage method corresponding to the output of the first operator; the second data group storage method supported by the second operator may refer to the data group storage method corresponding to the input of the second operator.

[0056] Optionally, after determining the first data packet storage method and the second data packet storage method, it can be determined whether the first data packet storage method and the second data packet storage method are the same. If the first data packet storage method and the second data packet storage method are different, then step 120 can be executed. If the first data packet storage method and the second data packet storage method are the same, then step 120 does not need to be executed.

[0057] Step 120: Based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, determine the target conversion mode information for converting the first data packet storage mode to the second data packet storage mode.

[0058] Alternatively, a neural network accelerator can be used to accelerate neural network models. A neural network accelerator may include a Brain Processing Unit (BPU).

[0059] It should be noted that the conversion of data block storage mode can be achieved by moving data in on-chip memory. Accordingly, the conversion operations supported by the neural network accelerator for data block storage mode conversion can include various types of data movement operations; where data movement can also be considered as data reordering. Therefore, it is possible to determine how to use various types of data movement operations to convert data from a first data block storage mode to a second data block storage mode in order to obtain the target conversion mode information.

[0060] It should be noted that the data stored in the on-chip memory (hereinafter referred to as target data) involved in the embodiments of this disclosure can be tensor data. Tensor data can include feature map data, and in addition, tensor data can also be regarded as a multidimensional array.

[0061] Step 130: Generate a conversion operator whose input is the output of the first operator, whose output is the input of the second operator, and which is used to convert the data grouping storage method according to the target conversion method information.

[0062] Assuming the first operator is operator A, the second operator is operator B, and the transformation operator is operator C, then the relationship between operators A, B, and C can be expressed as the following formula:

[0063] C_output = f1(A_output)

[0064] B_output = f2(C_output)

[0065] Where A_output represents the output of operator A, f1 represents the operation type corresponding to operator C (i.e., data grouping and storage method conversion according to target conversion method information), C_output represents the output of operator C, f2 represents the operation type corresponding to operator B, and B_output represents the output of operator B.

[0066] Step 140: Based on the operators and transformation operators in the neural network model, generate executable instructions corresponding to the neural network model through compilation.

[0067] Optionally, the compiler can perform compilation processing based on the various operators and transformation operators in the neural network model. Through compilation processing, an intermediate representation (IR) corresponding to the neural network model can be generated first, and then executable instructions corresponding to the neural network model can be generated based on the intermediate representation. The executable instructions corresponding to the neural network model can be computer-executable instructions. It is understood that the intermediate representation is one of the core data structures of the compiler, and it is a program representation between the source language and the target language during the compilation process. The specific compilation processing method in the embodiments of this disclosure can be adopted in any implementable manner according to actual needs, and this disclosure will not elaborate on it.

[0068] In the embodiments of this disclosure, the neural network model may involve two stages: a compilation stage and a runtime stage. During the compilation stage, for any two adjacent operators in the neural network model, such as the first operator and the second operator, if the first data group storage method supported by the first operator and the second operator support a different second data group storage method, a target conversion method information for converting the first data group storage method to the second data group storage method can be determined based on the conversion operation for data group storage method conversion supported by the neural network accelerator. A conversion operator is then generated, with the first operator's output as input and the second operator's input as output, and used to perform data group storage method conversion according to the target conversion method information. This conversion operator, along with the other operators in the neural network model, is used for the compilation processing of the neural network model. During the runtime stage, the executable instructions generated in the compilation stage are executed by the neural network accelerator, which is equivalent to running the neural network model on the neural network accelerator. In this process, after the output of the first operator is stored in the on-chip memory according to the first data group storage method, the conversion operator can perform data group storage method conversion on the output of the first operator in the on-chip memory according to the target conversion method information. The conversion result can be used as the input of the second operator. In this way, the input of the second operator can conform to the second data grouping storage method supported by the second operator, which helps to ensure the correct operation of the second operator, and thus ensures that the neural network model runs normally and reliably on the neural network accelerator.

[0069] In some optional examples, the data grouping storage method can represent the grouping storage of data in an on-chip memory that includes multiple partitions, where a single group corresponds to the size values ​​of multiple data dimensions and the traversal order of the multiple data dimensions; each group can be stored in a partition; each partition can include multiple slices, and each slice can include multiple storage locations.

[0070] Optionally, such as Figure 2As shown, the on-chip memory can include S partitions, such as partition 0 to partition S-1. Each partition can include M slices, such as slice 0 to slice M-1. Each slice can include N storage locations, such as storage location 0 to storage location N-1. In one example, M can be 16, N can be 16, and each storage location can store 1 byte. Thus, each slice can store 16 bytes, and each partition can store 256 bytes.

[0071] Optionally, width, height, and channel can each be considered as a data dimension.

[0072] Alternatively, the data grouping storage method involved in the embodiments of this disclosure may also be referred to as data layout.

[0073] In one example, the data group storage method can be 2h8w16c, which can also be written as [2h, 8w, (16c)]. This means that when data is grouped and stored in on-chip memory, a single group corresponds to a width of 8, a height of 2, and a channel of 16. The traversal order is: first traverse the channel, then traverse the width, and finally traverse the height.

[0074] In another example, the data can be stored in groups as 2h8c16w, which can also be written as [2h, 8c, (16w)]. This means that when storing data in groups in on-chip memory, each group corresponds to a width of 16, a height of 2, and a channel of 8. The traversal order is: first traverse the width, then the channel, and finally the height.

[0075] In another example, the data group storage method can be 4w4h16c, which can also be written as [4w, 4h, (16c)]. This means that when data is grouped and stored in on-chip memory, a single group corresponds to a width size of 4, a height size of 4, and a channel size of 16. The traversal order is: first traverse the channel, then traverse the height, and finally traverse the width.

[0076] In another example, the data can be stored in groups as 16w16c, which can also be written as [16w, (16c)]. This means that when storing data in groups in on-chip memory, each group corresponds to a width of 16 and a channel of 16. The traversal order is: first traverse the channel, then traverse the width.

[0077] Assuming the target data needs to be grouped and stored using the 2h8w16c data grouping storage method, the target data can be divided into several cuboids with a width of 8, a height of 2, and 16 channels (data padding can be added if necessary). Each cuboid can contain 256 values, each occupying one byte. Each cuboid can be considered a data block. For any cuboid, the 256 values ​​can be read in traversal order and placed into the same partition of on-chip memory in the order they are read, with each of the 256 values ​​occupying a storage location within that partition.

[0078] In the embodiments of this disclosure, the on-chip memory may include three storage levels: a partition level, a slice level, and a storage location level. The data grouping storage method can indicate the size values ​​of multiple data dimensions and the traversal order of these data dimensions. By referring to the size values ​​and traversal order indicated by the data grouping storage method, the target data can be grouped and stored in the on-chip memory with multiple storage levels, thereby improving the storage efficiency of the on-chip memory.

[0079] In some optional examples, the transformation operations supported by the neural network accelerator for data grouping and storage conversion may include:

[0080] The first type of transformation operation refers to data exchange across partitions.

[0081] The second type of transformation operation refers to transposing the slice dimension and the non-slice dimension within the partition.

[0082] The third type of transformation operation refers to the exchange of data between different slices within a partition.

[0083] Optionally, through the first type of transformation operation, data can be exchanged between partitions, and the granularity of the exchange can be a slice, with no modification required within the slice.

[0084] In one example, the transformation effect achievable by the first type of transformation operation can be seen by referring to... Figure 3-1 and Figure 3-2 ;in, Figure 3-1 This can represent the data grouping and storage method before performing the first type of transformation operation. Figure 3-2 This can represent the data grouping and storage method after performing the first type of transformation operation. Figure 3-1 The large rectangle below h0 and Figure 3-2 The large rectangle below w0-7 can be the same partition (e.g., partition 1) before and after performing the first type of transformation operation. Figure 3-1 The large rectangle below h1 and Figure 3-2The large rectangle below w8-15 can be the same partition (e.g., partition 2) before and after performing the first type of transformation operation. Figure 3-1 and Figure 3-2 It can be seen that, through the first type of transformation operation, the slice in the last half of partition 1 and the slice in the last half of partition 2 have swapped positions.

[0085] It should be noted that, Figure 3-1 The data grouping storage method in [16w(16c)] can be considered as [16w(16c)]. Figure 3-2 The data grouping storage method in [2h, 8w, (16c)] can be considered as [2h, 8w, (16c)]. The way to get [2h, 8w, (16c)] from [16w (16c)] can be understood as follows: first, write [16w (16c)] as [2w1, 8w0, (16c)], then add 2h before [2w1, 8w0, (16c)] to get 2h[2w1, 8w0, (16c)], then swap the positions of 2h and 2w1 in 2h[2w1, 8w0, (16c)] to get 2w1[2h, 8w0, (16c)]. [2h, 8w0, (16c)] is equivalent to [2h, 8w, (16c)], so we get [2h, 8w, (16c)].

[0086] Optionally, through the second type of transformation operation, the slice dimension and non-slice dimension can be transposed within the partition. That is, elements that were in the same slice before transposition are in the corresponding positions of different slices after transposition, and elements that were in the corresponding positions of different slices before transposition are in the same slice after transposition. Obviously, the slice can be modified.

[0087] In one example, the transformation effect achievable by the second type of transformation operation can be seen by referring to... Figure 4-1 and Figure 4-2 ;in, Figure 4-1 This can represent the data grouping and storage method before performing the second type of transformation operation. Figure 4-2 This can represent the data grouping and storage method after performing the second type of transformation operation.

[0088] It should be noted that, Figure 4-1 The data grouping storage method in [16w(16c)] can be considered as [16w(16c)]. Figure 4-2 The data grouping storage method in [16c(16w)] can be considered as [16c(16w)]. The way to obtain [16c(16w)] from [16w(16c)] can be understood as: swapping the positions of 16w and 16c in [16w(16c)].

[0089] Optionally, through the third type of transformation operation, data can be exchanged between different slices within a partition, while the data within a slice does not need to be modified.

[0090] In one example, the transformation effect of the third type of transformation operation can be referenced. Figure 5-1 and Figure 5-2 ;in, Figure 5-1 This can represent the data grouping and storage method before performing the first type of transformation operation. Figure 5-2 This can represent the data grouping and storage method after performing the first type of transformation operation.

[0091] It should be noted that, Figure 5-1 The data grouping storage method in [4h, 4w, (16c)] can be considered as [4h, 4w, (16c)]. Figure 5-2 The data grouping storage method in [4w, 4h, (16c)] can be considered as [4w, 4h, (16c)]. The way to obtain [4w, 4h, (16c)] from [4h, 4w, (16c)] can be considered as swapping the positions of 4h and 4w in [4h, 4w, (16c)].

[0092] In the embodiments of this disclosure, the first type of conversion operation can be used to realize cross-partition data exchange, while the second and third types of conversion operations can both be used to realize non-cross-partition data exchange. Since each partition can have a corresponding partition address, cross-partition data exchange can also be called cross-address data exchange, and non-cross-partition data exchange can also be called single-address data exchange. Through cross-address data exchange and single-address data exchange, data migration can be effectively realized, thereby achieving the conversion of data group storage method.

[0093] like Figure 6 The diagram shown is a flowchart illustrating a method for determining target conversion method information provided by some exemplary embodiments of this disclosure. Figure 6 The method shown may include steps 610 and 620. Optionally, a combination of steps 610 and 620 may be used as an alternative implementation of step 120 of this disclosure.

[0094] Step 610: Based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, determine at least one candidate conversion mode information for converting the first data packet storage mode into the second data packet storage mode.

[0095] In step 610, based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, all conversion mode information that can convert the first data packet storage mode into the second data packet storage mode can be found through brute force search. Each of these conversion mode information can be used as a candidate conversion mode information.

[0096] In some alternative embodiments of this disclosure, step 610 may include:

[0097] The first data group storage method is selected as the data group storage method to be used.

[0098] Filter the transformation operations to be used from the transformation operations supported by the neural network accelerator for data group storage mode conversion;

[0099] Determine the intermediate data group storage method obtained by performing the transformation operation on the data group storage method to be used;

[0100] Based on the matching relationship between the intermediate data group storage method and the second data group storage method, at least one candidate conversion method is determined.

[0101] Optionally, when filtering transformation operations to be used from those supported by the neural network accelerator for data group storage mode conversion, at least one type of transformation operation can be selected from the first type of transformation operation, the second type of transformation operation, and the third type of transformation operation, and each selected transformation operation can be used as a transformation operation to be used.

[0102] Optionally, the matching relationship between the intermediate data block storage method and the second data block storage method can be used to characterize whether the intermediate data block storage method and the second data block storage method match (i.e., whether they are the same).

[0103] If the intermediate data block storage method does not match the second data block storage method, the data block storage method to be used can be updated to the intermediate data block storage method, and the process can return to the step of filtering the conversion operation to be used from the conversion operation supported by the neural network accelerator for data block storage method conversion.

[0104] If the intermediate data block storage method matches the second data block storage method, the candidate conversion method information can be determined based on the conversion operations to be used executed sequentially during the process of obtaining the intermediate data block storage method from the first data block storage method.

[0105] In one example, the first data group storage method can be 16w16c. Therefore, 16w16c can be used as the starting point for searching for at least one candidate transformation method. Optionally, 16w16c can also be written as [16w, (16c)] or [16w1, 1w0, (16c)]. [16w, (16c)] can also be equivalently transformed into [2w1, 8w0, (16c)], [4w1, 4w0, (16c)], and [8w1, 2w0, (16c)]. When performing equivalent transformations, it is necessary to ensure that the coefficients preceding each w1 and each w0 are powers of 2, and that the product of the coefficients preceding w1 and w0 in the same transformation result is 16. Thus, the product of the coefficients preceding w1, w0, and c in the same transformation result can be 256, consistent with the amount of data that a single partition can hold.

[0106] Assuming the first data group storage method is used as the data group storage method to be used, the data group storage method to be used can be represented as [2w1, 8w0, (16c)], [4w1, 4w0, (16c)], [8w1, 2w0, (16c)] or [16w1, 1w0, (16c)].

[0107] For [2w1, 8w0, (16c)], 2h can be added before it to obtain 2h, [2w1, 8w0, (16c)]. Assuming the transformation operation to be used is a first-type transformation operation, the positions of 2h and 2w1 in 2h, [2w1, 8w0, (16c)] can be swapped to obtain 2w1, [2h, 8w0, (16c)]. [2h, 8w0, (16c)] can also be written as [2h, 8w, (16c)]. [2h, 8w, (16c)] can be used as an intermediate data group storage method, for example, called intermediate data group storage method 1.

[0108] For [4w1, 4w0, (16c)], 4h can be added before it to obtain 4h, [4w1, 4w0, (16c)]. Assuming the transformation operation to be used is a first-type transformation operation, the positions of 4h and 4w1 in 4h, [4w1, 4w0, (16c)] can be swapped to obtain 4w1, [4h, 4w0, (16c)]. [4h, 4w0, (16c)] can also be written as [4h, 4w, (16c)]. [4h, 4w, (16c)] can be used as an intermediate data group storage method, for example, called intermediate data group storage method 2.

[0109] For [8w1, 2w0, (16c)], 8h can be added before it to obtain 8h, [8w1, 2w0, (16c)]. Assuming the conversion operation to be used is a first-type conversion operation, the positions of 8h and 8w1 in 8h, [8w1, 2w0, (16c)] can be swapped to obtain 8w1, [8h, 2w0, (16c)]. [8h, 2w0, (16c)] can also be written as [8h, 2w, (16c)]. [8h, 2w, (16c)] can be used as an intermediate data group storage method, for example, called intermediate data group storage method 3.

[0110] For [16w1, 1w0, (16c)], 16h can be added before it to obtain 16h, [16w1, 1w0, (16c)]. Assuming the conversion operation to be used is a first-type conversion operation, the positions of 16h and 16w1 in 16h, [16w1, 1w0, (16c)] can be swapped to obtain 16w1, [16h, 1w0, (16c)]. [16h, 1w0, (16c)] can also be written as [16h, 1w, (16c)]. [16h, 1w, (16c)] can be used as an intermediate data group storage method, for example, called intermediate data group storage method 4.

[0111] For any of the intermediate data packet storage methods 1 to 4, the matching relationship between the intermediate data packet storage method and the second data packet storage method can be determined so as to determine the candidate conversion method information.

[0112] For example, for intermediate data group storage method 1, it can be determined whether intermediate data group storage method 1 is the same as the second data group storage method.

[0113] If intermediate data group storage method 1 is the same as the second data group storage method, then candidate conversion method information 1 can be determined. Candidate conversion method information 1 can indicate conversion through the first type of conversion operation, and the first type of conversion operation specifically is to exchange 2h and 2w1.

[0114] If the intermediate data group storage method 1 is different from the second data group storage method, the data group storage method to be used can be updated to the intermediate data group storage method 1, that is, updated to [2h, 8w, (16c)]. Next, the transformation operations to be used can be filtered again from the first type of transformation operation, the second type of transformation operation, and the third type of transformation operation. Optionally, in order to avoid duplicate searches caused by converting back to [16w, (16c)], when filtering the transformation operations to be used again, the second type of transformation operation and the third type of transformation operation can be treated as one transformation operation to be used.

[0115] Assuming the selected transformation operation is a second type of transformation operation, performing this operation on [2h, 8w, (16c)] will yield [16c, (2h, 8w)]. [16c, (2h, 8w)] can serve as an intermediate data group storage method, for example, referred to as intermediate data group storage method 5. If intermediate data group storage method 5 is the same as the second data group storage method, candidate transformation method information 2 can be determined. Candidate transformation method information 2 can be used to indicate that the transformation should first be performed using the first type of transformation operation, specifically by swapping 2h and 2w1, and then performing the transformation using the second type of transformation operation. If intermediate data group storage method 5 is different from the second data group storage method, the data group storage method to be used can be updated to intermediate data group storage method 5. Subsequent operations are similar to those performed after updating the data group storage method to be used to intermediate data group storage method 1 as described above, and will not be repeated here.

[0116] Assuming the selected transformation operation is a third type of transformation operation, performing this operation on [2h, 8w, (16c)] will yield [8w, 2h, (16c)]. [8w, 2h, (16c)] can serve as an intermediate data group storage method, for example, referred to as intermediate data group storage method 6. If intermediate data group storage method 6 is the same as the second data group storage method, candidate transformation method information 2 can be determined. Candidate transformation method information 2 can be used to indicate that the transformation should first be performed using the first type of transformation operation, specifically by swapping 2h and 2w1, and then performing the transformation using the third type of transformation operation. If intermediate data group storage method 6 is different from the second data group storage method, the data group storage method to be used can be updated to intermediate data group storage method 6. Subsequent operations are similar to those performed after updating the data group storage method to be used to intermediate data group storage method 1, and will not be repeated here.

[0117] The above details how to determine candidate transformation method information after obtaining intermediate data grouping storage method 1 through the first type of transformation operation. The method for determining candidate transformation method information after obtaining intermediate data grouping storage method 1, intermediate data grouping storage method 2, and intermediate data grouping storage method 3 through the first type of transformation operation is similar and will not be repeated here.

[0118] Furthermore, the above describes the situation where a first type of transformation operation is first performed on the first data group storage method, and candidate transformation method information is determined based on this. In some embodiments, a second type of transformation operation or a third type of transformation operation may also be performed on the first data group storage method first. For example, if the first data group storage method is 16w16c, then a second type of transformation operation is first performed on the first data group storage method to obtain [16c, (16w)], and then a first type of transformation operation can be performed on [16c, (16w)].

[0119] Thus, in this embodiment, a brute-force search can be used to find all candidate conversion methods that meet the requirements, thereby providing a very effective reference for determining the target conversion method.

[0120] Step 620: Select target conversion method information from at least one candidate conversion method information.

[0121] In some alternative embodiments of this disclosure, step 620 may include:

[0122] Based on the operation time corresponding to each type of conversion operation for data group storage mode conversion supported by neural network accelerator, the estimated conversion time corresponding to each of at least one candidate conversion mode information is determined.

[0123] From at least one candidate conversion method, select the candidate conversion method with the shortest estimated conversion time;

[0124] Based on the candidate conversion method information with the shortest estimated conversion time, the target conversion method information is determined.

[0125] Optionally, the operation durations for the first, second, and third types of transformation operations can be determined in advance through experiments. The operation duration for the first type of transformation operation can be denoted as t1, the operation duration for the second type of transformation operation can be denoted as t2, and the operation duration for the third type of transformation operation can be denoted as t3.

[0126] In one example, the at least one candidate transformation method information determined by executing step 610 specifically includes two candidate transformation method information. The execution order of the transformation operations indicated by the first candidate transformation method information is: first type of transformation operation → second type of transformation operation → third type of transformation operation. The execution order of the transformation operations indicated by the second candidate transformation method information is: third type of transformation operation → second type of transformation operation → first type of transformation operation → third type of transformation operation. Therefore, the estimated transformation time corresponding to the first candidate transformation method information can be expressed as: T1 = k1*t1 + t2 + t3, and the estimated transformation time corresponding to the second candidate transformation method information can be expressed as: T2 = t3 + t2 + k2*t1 + t3. Optionally, the values ​​of k1 and k2 can be related to the exchange coefficients involved in the first type of transformation operation. For example, the first type of transformation operation corresponding to k1 involves the exchange of 8h and 8w1, and the first type of transformation operation corresponding to k2 involves the exchange of 4h and 4w1. Since 8 is greater than 4, k1 can be set greater than k2. For example, the first type of transformation operation corresponding to k1 involves the exchange of 8h and 8w1, and the first type of transformation operation corresponding to k2 involves the exchange of 16h and 16w1. Since 8 is less than 16, we can make k1 less than k2.

[0127] After determining the estimated conversion time for each of the at least one candidate conversion method, the estimated conversion times for each candidate conversion method can be compared to select the candidate conversion method with the shortest estimated conversion time. Then, the candidate conversion method with the shortest estimated conversion time can be directly used as the target conversion method.

[0128] In some embodiments, there may be several candidate conversion methods whose estimated conversion times are tied for the shortest. For example, if the at least one candidate conversion method determined by step 610 is specifically four candidate conversion methods, and two of these candidate conversion methods have the shortest estimated conversion times, then the number of conversion operations required to perform the data grouping storage method conversion according to these two candidate conversion methods can be determined. If the number of operations corresponding to these two candidate conversion methods is different, then the candidate conversion method with the fewest operations can be selected as the target conversion method. If the number of operations corresponding to these two candidate conversion methods is the same, then one of these two candidate conversion methods can be randomly selected as the target conversion method. In one example, among the two candidate conversion methods, the execution order of the conversion operations indicated by the first candidate conversion method is: first type of conversion operation → second type of conversion operation → third type of conversion operation, while the execution order of the conversion operations indicated by the second candidate conversion method is: third type of conversion operation → second type of conversion operation → first type of conversion operation → third type of conversion operation. Obviously, the first candidate conversion method requires three conversion operations, while the second candidate conversion method requires four conversion operations. The first candidate conversion method requires fewer conversion operations, so the first candidate conversion method can be identified as the target conversion method.

[0129] In this embodiment, the operation time corresponding to the first type of conversion operation, the second type of conversion operation, and the third type of conversion operation can be referenced to reasonably estimate the conversion time corresponding to each of the at least one candidate conversion method information. The estimated conversion time can provide a very effective reference for determining the target conversion method information, which is conducive to ensuring the rationality of the target conversion method information, thereby helping to ensure the conversion efficiency from the first data group storage method to the second data group storage method.

[0130] Of course, the implementation of step 620 is not limited to this. For example, by executing at least one candidate conversion method information determined in step 610, specifically four candidate conversion method information, it is possible to determine the number of conversion operations that need to be performed when converting the data group storage method for these four candidate conversion method information, and select one of the four candidate conversion method information as the target conversion method information based on the size relationship between the number of times corresponding to each of the four candidate conversion method information.

[0131] In the embodiments of this disclosure, referring to the conversion operation for data packet storage mode conversion supported by the neural network accelerator, all conversion mode information that can convert the first data packet storage mode into the second data packet storage mode can be found, and suitable conversion mode information can be selected as the target conversion mode information. For example, the target conversion mode information can be made to bypass the very inefficient data packet storage mode as much as possible. In this way, it is beneficial to ensure the rationality of the target conversion mode information, achieve efficient data rearrangement, and ensure the conversion efficiency of data packet storage mode.

[0132] like Figure 7 The diagram shown is a flowchart illustrating a method for determining target conversion method information provided by some other exemplary embodiments of this disclosure. Figure 7 The method shown may include steps 710, 720, and 730. Optionally, a combination of steps 710, 720, and 730 may be used as an alternative implementation of step 120 of this disclosure.

[0133] Step 710: Obtain the target correspondence based on the conversion operation for data group storage mode conversion supported by the neural network accelerator; wherein, the target correspondence refers to the correspondence between the source data group storage mode, the destination data group storage mode, and the reference conversion mode information from the source data group storage mode to the destination data group storage mode.

[0134] Generally, the data grouping and storage methods that operators in a neural network model may involve are limited. For example, if there are a total of R data grouping and storage methods, then by using one of the R data grouping and storage methods as the source data grouping and storage method, and another of the R data grouping and storage methods as the destination data grouping and storage method, we can obtain R*R combinations of storage methods. For each of the R*R combinations of storage methods, we can pre-determine... Figure 6 The method described in the illustrated embodiment determines at least one candidate conversion method that can convert the source data packet storage method in the storage method combination to the destination data packet storage method in the storage method combination. The candidate conversion method with the shortest estimated conversion time among the determined candidate conversion method information is then used as the reference conversion method information corresponding to the storage method combination. Based on the R data packet storage methods and the reference conversion method information corresponding to each of the R data packet storage methods, the target correspondence involved in step 710 can be constructed. Optionally, the local information carried by the target correspondence can be seen in Table 1 below:

[0135] Source data group storage method destination data group storage method Reference conversion method information Data group storage method x1 Data group storage method y1 Conversion method information z1 Data group storage method x2 Data group storage method y2 Conversion method information z2 Data group storage method x3 Data group storage method y3 Conversion method information z3

[0136] Table 1

[0137] Step 720: Using the source data group storage method as the first data group storage method and the destination data group storage method as the second data group storage method as search conditions, search for the corresponding reference conversion method information in the target correspondence.

[0138] Assuming the first data group storage method is data group storage method x2 and the second data group storage method is data group storage method y2, then by referring to Table 1, we can see that the corresponding reference conversion method information in the target correspondence is conversion method information z2.

[0139] Step 730: Determine the target conversion method information based on the searched reference conversion method information.

[0140] Optionally, the searched reference conversion method information can be directly used as the target conversion method information.

[0141] In the embodiments of this disclosure, a target correspondence can be pre-constructed. In this way, after determining the first data group storage method supported by the first operator and the second data group storage method supported by the second operator, the target transformation method information can be determined efficiently and quickly by searching in the target correspondence, which is beneficial to ensuring the compilation efficiency of the neural network model.

[0142] like Figure 8 The diagram shown is a flowchart illustrating a method for generating transformation operators provided by some exemplary embodiments of this disclosure. Figure 8 The method shown may include steps 810, 820, and 830. Optionally, a combination of steps 810, 820, and 830 may be used as an alternative implementation of step 130 of this disclosure.

[0143] Step 810: Based on the target transformation method information, determine multiple intermediate data group storage methods arranged in sequence; wherein each intermediate data group storage method corresponds to a type of transformation operation supported by the neural network accelerator for data group storage method transformation.

[0144] Optionally, the target conversion method information can be used to indicate which conversion operations are required to transform from the first data packet storage method to the second data packet storage method, in what order these conversion operations are performed, and what intermediate data packet storage method is obtained from each type of conversion operation. In this way, multiple intermediate data packet storage methods arranged in sequence can be determined based on the target conversion method information.

[0145] In one example, the first data group storage method can be 16w2h8c, and the second data group storage method can be 16w16c. The target transformation method information can be used to indicate that [16w, (2h, 8c)] is first transformed into [2h, 8c, (16w)] through the second type of transformation operation, then [2h, 8c, (16w)] is transformed into [2c1, 8c0, (16w)] through the first type of transformation operation, and finally [2c1, 8c0, (16w)] is transformed into [16w, (2c1, 8c0)] through the second type of transformation operation; where [16w, (2c1, 8c0)] is equivalent to 16w16c. In this way, based on the target transformation method information, the three intermediate data group storage methods arranged in sequence can be determined. If these three intermediate data grouping storage methods are represented as intermediate data grouping storage method r1, intermediate data grouping storage method r2, and intermediate data grouping storage method r3 respectively, then intermediate data grouping storage method r1 can be [2h, 8c, (16w)], intermediate data grouping storage method r2 can be [2c1, 8c0, (16w)], and intermediate data grouping storage method r3 can be [16w, (2c1, 8c0)]. Furthermore, the conversion operation corresponding to intermediate data grouping storage method r1 can be a second type of conversion operation, the conversion operation corresponding to intermediate data grouping storage method r2 can be a first type of conversion operation, and the conversion operation corresponding to intermediate data grouping storage method r3 can be a second type of conversion operation.

[0146] Optionally, when converting [16w, (2h, 8c)] to [2h, 8c, (16w)] using the second type of conversion operation, the conversion effect can be seen in [link to documentation]. Figures 9-1 to 9-2 ;in, Figure 9-1 Corresponding to [16w, (2h, 8c)], Figure 9-2 This corresponds to [2h, 8c, (16w)]. When converting [2h, 8c, (16w)] to [2c1, 8c0, (16w)] using the first type of transformation operation, the transformation effect can be seen in […]. Figures 9-3 to 9-4 ;in, Figure 9-3 Corresponding to [2h, 8c, (16w)], Figure 9-4 This corresponds to [2c1, 8c0, (16w)]. When converting [2c1, 8c0, (16w)] to [16w, (2c1, 8c0)] using the second type of transformation operation, the conversion effect can be seen in [link to documentation]. Figures 9-5 to 9-6 ;in, Figure 9-5 Corresponding to [2c1, 8c0, (16w)], Figure 9-6 It corresponds to [16w, (2c1, 8c0)].

[0147] Step 820: Generate a third operator corresponding to each of the multiple intermediate data grouping storage methods; wherein, the input of the third operator corresponding to the first intermediate data grouping storage method is the output of the first operator, the output of the third operator corresponding to the last intermediate data grouping storage method is the input of the second operator, and the input of the third operator corresponding to the intermediate data grouping storage method that is not first in the order is the output of the third operator corresponding to the previous intermediate data grouping storage method.

[0148] In step 820, third operators corresponding to intermediate data grouping storage methods r1, r2, and r3 can be generated. The input of the third operator corresponding to intermediate data grouping storage method r1 is the output of the first operator; the output of the third operator corresponding to intermediate data grouping storage method r1 is the input of the third operator corresponding to intermediate data grouping storage method r2; the output of the third operator corresponding to intermediate data grouping storage method r2 is the input of the third operator corresponding to intermediate data grouping storage method r3; and the output of the third operator corresponding to intermediate data grouping storage method r3 is the input of the second operator.

[0149] Step 830: Determine the transformation operator based on the third operator corresponding to each of the multiple intermediate data grouping and storage methods.

[0150] Optionally, the transformation operator may include a third operator corresponding to each of the multiple intermediate data grouping storage methods. For example, the transformation operator may include a third operator corresponding to each of the intermediate data grouping storage methods r1, r2, and r3. Thus, during the runtime phase, the data flow from the first operator to the second operator can be referenced... Figure 10 .

[0151] It should be noted that the conversion from the first data block storage method to the second data block storage method may involve multiple steps, each step performing a type of conversion operation. Therefore, a corresponding third operator can be generated for each step. By combining these third operators, a conversion operator can be obtained efficiently and reliably. This conversion operator effectively realizes the conversion from the first data block storage method to the second data block storage method, ensuring that the input of the second operator conforms to the second data block storage method it supports. This helps ensure the correct operation of the second operator, and consequently, ensures the normal and reliable operation of the neural network model on the neural network accelerator.

[0152] In one example, the batch size, height, width, and number of channels of the target data are 1, 30, 10, and 7, respectively. If the first data group is stored in the form of 2h16w8c, since 2h16w8c can be written as [2h, 8w, (2w, 8c)], and [2h, 8w, (2w, 8c)] can be further written as [2h, 8w1, (2w0, 8c)], the target data can be represented as follows: 15h[2h, 8w1, (2w0, 8c)]. Here, 15h indicates that h of size 30 is divided into 15 groups and stored in 15 partition addresses. 2h indicates that each partition address contains 2 groups of h. 8w1 indicates that w is first aligned to 16 parts and then divided equally into 8 parts. 2w0 indicates that each slice contains 2 evenly divided w. 8c indicates that each w0 contains 8 c. When storing the target data in groups according to the first data grouping storage method, each of the 15 partition addresses can store 256 values. Specifically, the valid portion of the partition corresponding to the first partition address consists of all values ​​in the target data with h indices 0-1, w indices 0-9, and c indices 0-7. The valid portion of the partition corresponding to the second partition address consists of all values ​​in the target data with h indices 2-3, w indices 0-9, and c indices 0-7. The valid portion of the partition corresponding to the third partition address consists of all values ​​in the target data with h indices 4-5, w indices 0-9, and c indices 0-7. ... The valid portion of the partition corresponding to the fifteenth partition address consists of all values ​​in the target data with h indices 28-29, w indices 0-9, and c indices 0-7. It should be noted that the h, w, and c indices can be sorted along the height direction, the width direction, and the channel direction, respectively.

[0153] During the compilation phase, given the first and second data grouping storage methods, the target transformation method information can be determined through brute-force search based on the first, second, and third types of transformation operations. Alternatively, it can be determined based on a pre-constructed target correspondence based on the first, second, and third types of transformation operations. Next, transformation operators can be generated based on the target transformation method information, and executable instructions corresponding to the neural network model can be generated through compilation processing based on the various operators in the neural network model and the transformation operators. Thus, during the computation phase, the transformation operator can achieve the transformation of the data grouping storage method through at least some of the first, second, and third types of transformation operations, ensuring that the input of the second operator conforms to the second data grouping storage method supported by the second operator.

[0154] In some embodiments, the second operator may support multiple second data grouping storage methods, meaning the second operator does not have strict requirements on the data grouping storage method. Therefore, the compiler can select the most suitable second data grouping storage method from among the multiple methods based on the size values ​​of each dimension of the target data (i.e., batch size, height, width, and number of channels), and only use the selected method to determine the target transformation method information. This minimizes data padding and space overhead, improves the transformation efficiency of the data grouping storage method, and reduces global data rearrangement overhead.

[0155] In summary, the embodiments of this disclosure can achieve relatively complex data grouping and storage method conversion with low hardware complexity, thereby ensuring that the neural network model runs normally and reliably on the neural network accelerator. Furthermore, the embodiments of this disclosure can also minimize the inefficiency and space overhead caused by data expansion.

[0156] Exemplary device

[0157] Figure 11 This is a schematic diagram of the structure of an apparatus for compiling a neural network model to obtain executable instructions, provided by some exemplary embodiments of the present disclosure. Figure 11 The apparatus shown includes:

[0158] The first determining module 1110 is used to determine the first data grouping storage method supported by the first operator and the second data grouping storage method supported by the second operator in the neural network model; wherein, the first operator is the operator preceding the second operator;

[0159] The second determining module 1120 is used to determine, based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, the target conversion mode information for converting the first data packet storage mode determined by the first determining module 1110 into the second data packet storage mode determined by the first determining module 1110.

[0160] The first generation module 1130 is used to generate a conversion operator whose input is the output of the first operator, whose output is the input of the second operator, and whose conversion operator is used to perform data grouping and storage method conversion according to the target conversion method information determined by the second determination module 1120.

[0161] The second generation module 1140 is used to generate executable instructions corresponding to the neural network model through compilation processing based on the operators in the neural network model and the transformation operators generated by the first generation module 1130.

[0162] In some optional examples, such as Figure 12 As shown, the second determining module 1120 includes:

[0163] The first determining submodule 11201 is used to determine at least one candidate conversion method information for converting the first data block storage method determined by the first determining module 1110 into the second data block storage method determined by the first determining module 1110, based on the conversion operation for data block storage method conversion supported by the neural network accelerator.

[0164] The filtering submodule 11203 is used to filter target conversion method information from at least one candidate conversion method information determined by the first determining submodule 11201.

[0165] In some optional examples, the first determined submodule 11201 includes:

[0166] The first determining unit is used to select the first data group storage method determined by the first determining module 1110 as the data group storage method to be used.

[0167] The first filtering unit is used to filter the transformation operations to be used from the transformation operations supported by the neural network accelerator for data group storage mode conversion;

[0168] The second determining unit is used to determine the intermediate data grouping storage method obtained by performing the conversion operation to be used by the first filtering unit on the data grouping storage method to be used.

[0169] The third determining unit is used to determine at least one candidate conversion method information based on the matching relationship between the intermediate data group storage method determined by the second determining unit and the second data group storage method determined by the first determining module 1110.

[0170] In some optional examples, the filtering submodule 11203 includes:

[0171] The fourth determining unit is used to determine the estimated conversion time corresponding to each of the at least one candidate conversion method information determined by the first determining submodule 11201, based on the operation time corresponding to each type of conversion operation for data group storage mode conversion supported by the neural network accelerator.

[0172] The second filtering unit is used to filter the candidate conversion method information with the shortest estimated conversion time from at least one candidate conversion method information determined by the first determining submodule 11201.

[0173] The fifth determining unit is used to determine the target conversion method information based on the candidate conversion method information with the shortest estimated conversion time selected by the second filtering unit.

[0174] In some optional examples, such as Figure 13 As shown, the second determining module 1120 includes:

[0175] The acquisition submodule 11205 is used to acquire the target correspondence based on the conversion operation for data group storage mode conversion supported by the neural network accelerator; wherein, the target correspondence refers to the correspondence between the source data group storage mode, the destination data group storage mode, and the reference conversion mode information from the data group storage mode to the destination data group storage mode;

[0176] Search submodule 11207 is used to search for corresponding reference conversion method information in the target correspondence obtained by acquisition submodule 11205, with the source data group storage method being the first data group storage method determined by the first determination module 1110 and the destination data group storage method being the second data group storage method determined by the first determination module 1110 as search conditions.

[0177] The second determining submodule 11209 is used to determine the target conversion method information based on the reference conversion method information searched by the search submodule 11207.

[0178] In some optional examples, such as Figure 14 As shown, the first generation module 1130 includes:

[0179] The third determining submodule 11301 is used to determine multiple intermediate data group storage methods arranged in sequence based on the target transformation method information determined by the second determining module 1120; wherein, each intermediate data group storage method corresponds to a type of transformation operation supported by the neural network accelerator for data group storage method transformation;

[0180] The generation submodule 11303 is used to generate the third operators corresponding to each of the multiple intermediate data grouping storage methods determined by the third determination submodule 11301; wherein, the input of the third operator corresponding to the intermediate data grouping storage method that is ranked first is the output of the first operator, the output of the third operator corresponding to the intermediate data grouping storage method that is ranked last is the input of the second operator, and the input of the third operator corresponding to the intermediate data grouping storage method that is not ranked first is the output of the third operator corresponding to the previous intermediate data grouping storage method;

[0181] The fourth determining submodule 11305 is used to determine the transformation operator based on the third operator corresponding to each of the multiple intermediate data grouping storage methods generated by the generating submodule 11303.

[0182] In some optional examples, the data grouping storage method means that when data is stored in groups in an on-chip memory that includes multiple partitions, each group corresponds to the size values ​​of multiple data dimensions and the traversal order of the multiple data dimensions; each group is stored in a partition; each partition includes multiple slices, and each slice includes multiple storage locations.

[0183] In some optional examples, the neural network accelerator supports the following conversion operations for data grouping and storage mode conversion:

[0184] The first type of transformation operation refers to data exchange across partitions.

[0185] The second type of transformation operation refers to transposing the slice dimension and the non-slice dimension within the partition.

[0186] The third type of transformation operation refers to the exchange of data between different slices within a partition.

[0187] In the apparatus disclosed herein, the various optional embodiments, optional implementation methods and optional examples disclosed above can be flexibly selected and combined as needed to achieve the corresponding functions and effects, and this disclosure does not list them all.

[0188] Exemplary electronic devices

[0189] Figure 15 The illustration shows a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 1500 includes one or more processors 1510 and memory 1520.

[0190] The processor 1510 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1500 to perform desired functions.

[0191] The memory 1520 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1510 may execute one or more computer program instructions to implement the methods of the various embodiments of this disclosure described above and / or other desired functions.

[0192] In one example, the electronic device 1500 may also include an input device 1530 and an output device 1540, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0193] The input device 1530 may also include, for example, a keyboard, a mouse, etc.

[0194] The output device 1540 can output various information to the outside, including, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0195] Of course, for the sake of simplicity, Figure 15 Only some of the components of the electronic device 1500 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 1500 may include any other suitable components depending on the specific application.

[0196] Exemplary computer program products and computer-readable storage media

[0197] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0198] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0199] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0200] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0201] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. The specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the specific details described above.

[0202] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for compiling a neural network model to obtain executable instructions, comprising: Determine the first data grouping storage method supported by the first operator and the second data grouping storage method supported by the second operator in the neural network model; wherein, the first operator is the preceding operator of the second operator; Based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, the target conversion mode information for converting the first data packet storage mode to the second data packet storage mode is determined; A conversion operator is generated whose input is the output of the first operator, whose output is the input of the second operator, and is used to convert the data grouping storage method according to the target conversion method information; Based on the operators and transformation operators in the neural network model, executable instructions corresponding to the neural network model are generated through compilation. The conversion operation based on neural network accelerator support for data packet storage mode conversion determines the target conversion mode information for converting the first data packet storage mode to the second data packet storage mode, including: Based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, at least one candidate conversion mode information for converting the first data packet storage mode to the second data packet storage mode is determined; and the target conversion mode information is filtered from the at least one candidate conversion mode information. The conversion operation based on neural network accelerator-supported for data packet storage mode conversion determines at least one candidate conversion mode information for converting the first data packet storage mode to the second data packet storage mode, including: The first data group storage method is selected as the data group storage method to be used. Filter the transformation operations to be used from the transformation operations supported by the neural network accelerator for data group storage mode conversion; Determine the intermediate data group storage method obtained by performing the transformation operation on the data group storage method to be used; Based on the matching relationship between the intermediate data group storage method and the second data group storage method, at least one candidate conversion method is determined.

2. The method according to claim 1, wherein, The step of filtering the target conversion method information from at least one of the candidate conversion method information includes: Based on the operation time corresponding to each type of conversion operation for data group storage mode conversion supported by the neural network accelerator, the estimated conversion time corresponding to each of at least one of the candidate conversion mode information is determined. From at least one of the candidate conversion method information, filter the candidate conversion method information with the shortest estimated conversion time; Based on the candidate conversion method information with the shortest estimated conversion time, the target conversion method information is determined.

3. The method according to claim 1, wherein, The conversion operator, whose input is the output of the first operator and whose output is the input of the second operator, and which is used to convert the data grouping storage method according to the target conversion method information, includes: Based on the target transformation method information, a plurality of intermediate data group storage methods are determined in sequence; wherein each of the intermediate data group storage methods corresponds to a type of transformation operation supported by the neural network accelerator for data group storage method transformation; Generate a third operator corresponding to each of the intermediate data grouping storage methods; wherein, the input of the third operator corresponding to the intermediate data grouping storage method that is first in the order is the output of the first operator, the output of the third operator corresponding to the intermediate data grouping storage method that is last in the order is the input of the second operator, and the input of the third operator corresponding to the intermediate data grouping storage method that is not first in the order is the output of the third operator corresponding to the previous intermediate data grouping storage method; The transformation operator is determined based on the third operator corresponding to each of the multiple intermediate data grouping and storage methods.

4. The method according to any one of claims 1-3, wherein, The data grouping storage method refers to the grouping storage of data in an on-chip memory that includes multiple partitions. Each group corresponds to the size values ​​of multiple data dimensions and the traversal order of the multiple data dimensions. Each group is stored in one partition. Each partition includes multiple slices, and each slice includes multiple storage locations.

5. The method according to claim 4, wherein, The neural network accelerator supports the following conversion operations for data group storage mode conversion: The first type of transformation operation refers to: data exchange across the partition; The second type of transformation operation refers to transposing the slice dimension and the non-slice dimension within the partition. The third type of transformation operation refers to the data exchange between different slices within the partition.

6. A method for compiling a neural network model to obtain executable instructions, comprising: Determine the first data grouping storage method supported by the first operator and the second data grouping storage method supported by the second operator in the neural network model; wherein, the first operator is the preceding operator of the second operator; Based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, the target conversion mode information for converting the first data packet storage mode to the second data packet storage mode is determined; A conversion operator is generated whose input is the output of the first operator, whose output is the input of the second operator, and is used to convert the data grouping storage method according to the target conversion method information; Based on the operators and transformation operators in the neural network model, executable instructions corresponding to the neural network model are generated through compilation. The conversion operation based on neural network accelerator support for data packet storage mode conversion determines the target conversion mode information for converting the first data packet storage mode to the second data packet storage mode, including: Obtain the target correspondence based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator; wherein, the target correspondence refers to the correspondence between the source data packet storage mode, the destination data packet storage mode, and the reference conversion mode information from the data packet storage mode to the destination data packet storage mode; Using the source data group storage method as the first data group storage method and the destination data group storage method as the second data group storage method as search conditions, the corresponding reference conversion method information is searched in the target correspondence; Based on the searched reference conversion method information, the target conversion method information is determined.

7. The method according to claim 6, wherein, The conversion operator, whose input is the output of the first operator and whose output is the input of the second operator, and which is used to convert the data grouping storage method according to the target conversion method information, includes: Based on the target transformation method information, a plurality of intermediate data group storage methods are determined in sequence; wherein each of the intermediate data group storage methods corresponds to a type of transformation operation supported by the neural network accelerator for data group storage method transformation; Generate a third operator corresponding to each of the intermediate data grouping storage methods; wherein, the input of the third operator corresponding to the intermediate data grouping storage method that is first in the order is the output of the first operator, the output of the third operator corresponding to the intermediate data grouping storage method that is last in the order is the input of the second operator, and the input of the third operator corresponding to the intermediate data grouping storage method that is not first in the order is the output of the third operator corresponding to the previous intermediate data grouping storage method; The transformation operator is determined based on the third operator corresponding to each of the multiple intermediate data grouping and storage methods.

8. The method according to claim 6 or 7, wherein, The data grouping storage method refers to the grouping storage of data in an on-chip memory that includes multiple partitions. Each group corresponds to the size values ​​of multiple data dimensions and the traversal order of the multiple data dimensions. Each group is stored in one partition. Each partition includes multiple slices, and each slice includes multiple storage locations.

9. The method according to claim 8, wherein, The neural network accelerator supports the following conversion operations for data group storage mode conversion: The first type of transformation operation refers to: data exchange across the partition; The second type of transformation operation refers to transposing the slice dimension and the non-slice dimension within the partition. The third type of transformation operation refers to the data exchange between different slices within the partition.

10. An apparatus for compiling a neural network model to obtain executable instructions, comprising: The first determining module is used to determine the first data grouping storage method supported by the first operator and the second data grouping storage method supported by the second operator in the neural network model; wherein, the first operator is the preceding operator of the second operator; The second determining module is used to determine, based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, the target conversion mode information for converting the first data packet storage mode determined by the first determining module into the second data packet storage mode determined by the first determining module. The first generation module is used to generate a conversion operator whose input is the output of the first operator, whose output is the input of the second operator, and whose conversion operator is used to perform data grouping storage method conversion according to the target conversion method information determined by the second determining module; The second generation module is used to generate executable instructions corresponding to the neural network model through compilation processing based on the operators in the neural network model and the transformation operators generated by the first generation module. The second determining module includes: A first determining submodule is configured to determine at least one candidate conversion method information for converting the first data block storage method determined by the first determining module into the second data block storage method determined by the first determining module, based on a conversion operation for data block storage method conversion supported by a neural network accelerator; and a filtering submodule is configured to filter the target conversion method information from the at least one candidate conversion method information determined by the first determining submodule. The first determining submodule includes: The first determining unit is used to select the first data group storage method as the data group storage method to be used. The first filtering unit is used to filter the transformation operations to be used from the transformation operations supported by the neural network accelerator for data group storage mode conversion; The second determining unit is used to determine the intermediate data group storage method obtained by performing the conversion operation on the data group storage method to be used; The third determining unit is used to determine at least one of the candidate conversion method information based on the matching relationship between the intermediate data group storage method and the second data group storage method.

11. An apparatus for compiling a neural network model to obtain executable instructions, comprising: The first determining module is used to determine the first data grouping storage method supported by the first operator and the second data grouping storage method supported by the second operator in the neural network model; wherein, the first operator is the preceding operator of the second operator; The second determining module is used to determine, based on the conversion operation for data packet storage mode conversion supported by the neural network accelerator, the target conversion mode information for converting the first data packet storage mode determined by the first determining module into the second data packet storage mode determined by the first determining module. The first generation module is used to generate a conversion operator whose input is the output of the first operator, whose output is the input of the second operator, and whose conversion operator is used to perform data grouping storage method conversion according to the target conversion method information determined by the second determining module; The second generation module is used to generate executable instructions corresponding to the neural network model through compilation processing based on the operators in the neural network model and the transformation operators generated by the first generation module. The second determining module includes: The acquisition submodule is used to acquire the target correspondence based on the conversion operation for data group storage mode conversion supported by the neural network accelerator; wherein, the target correspondence refers to the correspondence between the source data group storage mode, the destination data group storage mode, and the reference conversion mode information from the data group storage mode to the destination data group storage mode; The search submodule is used to search for the corresponding reference conversion method information in the target correspondence using the source data group storage method as the first data group storage method and the destination data group storage method as the second data group storage method as search conditions; The second determining submodule is used to determine the target conversion method information based on the searched reference conversion method information.

12. A computer-readable storage medium storing a computer program for performing the method of compiling a neural network model to obtain executable instructions as described in any one of claims 1-5 or 6-9.

13. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for compiling a neural network model to obtain executable instructions as described in any one of claims 1-5 or 6-9.

Images