Compiler optimization method, electronic device and storage medium

Abstract

A compiler optimization method, electronic device and storage medium. The compiler optimization method includes: using a compiler to compile input data to obtain compiled data, wherein the compiled data contains a first operator; using a first conversion function to configure the first operator, wherein the first conversion function is used to convert a first storage space access mode supported by the first operator into a second storage space access mode; based on the first conversion function, modifying the calling method of the first operator in the compiler to convert the first operator into a second operator, wherein the second operator supports the second storage space access mode. The compiler optimization method expands the types of data access modes supported by high-performance compilers in deep learning frameworks, and does not require calling conversion operators for additional tensor format conversions, thereby improving the execution efficiency of computing tasks and improving the performance of artificial intelligence chips.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a compiler optimization method, an electronic device, and a storage medium. Background Art

[0002] In the field of artificial intelligence (AI), deep neural networks (DNNs) have become the foundation of advanced technologies and the core driving force of many applications. With the continuous generalization of deep learning application scenarios, computing tasks performed by deep neural networks need to be deployed on different computing devices and hardware architectures. Actual deployment or training scenarios often have more radical requirements for the performance of deep neural networks, such as the need to customize computing codes based on the characteristics of the deployed hardware. These requirements are difficult to meet in general AI frameworks. Using AI compilers to compile and optimize deep learning computing tasks can meet the hardware performance requirements of actual deployment or training scenarios of computing tasks. Summary of the invention

[0003] At least one embodiment of the present disclosure provides a compiler optimization method, which includes: using the compiler to compile input data to obtain compiled data, wherein the compiled data includes a first operator; using a first conversion function to configure the first operator, wherein the first conversion function is used to convert a first storage space access mode supported by the first operator into a second storage space access mode; based on the first conversion function, modifying the calling method of the first operator in the compiler to convert the first operator into a second operator, wherein the second operator supports the second storage space access mode.

[0004] For example, in the optimization method of the compiler provided in at least one embodiment of the present disclosure, the target storage space to be accessed includes multiple storage units, and the target storage space includes an offset address, and the offset address is used to determine the target storage unit to be accessed among the multiple storage units, and the use of the first conversion function to configure the first operator includes: setting the first conversion function based on the second storage space access mode, wherein the first conversion function is used to modify the offset address of the target storage space corresponding to the first storage space access mode to the offset address corresponding to the second storage space access mode; adding the first conversion function in the first operator.

[0005] For example, in the compiler optimization method provided by at least one embodiment of the present disclosure, the first operator includes a first address pointer for accessing the target storage space, the first address pointer corresponds to the first storage space access mode, and the method of modifying the calling method of the first operator in the compiler based on the first conversion function to convert the first operator into a second operator includes: calling the first conversion function in the first operator in the compiler to modify the first address pointer in the first operator to the second address pointer in the second operator, wherein the second address pointer corresponds to the second storage space access mode.

[0006] For example, in the optimization method of the compiler provided in at least one embodiment of the present disclosure, the target storage space includes multiple storage units, the target storage space includes an offset address, the offset address is used to determine the target storage unit to be accessed among the multiple storage units, the first address pointer is calculated from the offset address corresponding to the first storage space access mode, and the second address pointer is calculated from the offset address corresponding to the second storage space access mode.

[0007] For example, in the compiler optimization method provided in at least one embodiment of the present disclosure, the target storage space to be accessed includes multiple storage units, the target storage space includes a first address, the first address is used to determine the mapping relationship between the multiple storage units and the multiple processing units, the multiple processing units are used to access the target storage space, and the modification of the calling method of the first operator in the compiler to convert the first operator into a second operator also includes: modifying the first address of the target storage space defined in the first operator to the first address of the target storage space defined in the second operator in the compiler to modify the mapping relationship between the multiple storage units and the multiple processing units, wherein the modified mapping relationship between the multiple storage units and the multiple processing units supports the second storage space access mode.

[0008] For example, in the compiler optimization method provided by at least one embodiment of the present disclosure, the multiple processing units include a first processing unit to an N-th processing unit, the multiple storage units include a first storage unit to an N-th storage unit, wherein N is an integer greater than 1, and in the second storage space access mode, the k-th processing unit among the multiple processing units is configured to access the k-th storage unit among the multiple storage units, wherein k = 1, 2,…, N.

[0009] For example, in the compiler optimization method provided in at least one embodiment of the present disclosure, each of the multiple storage units corresponds to at least one program block, and in the mapping relationship determined by the first address defined in the second operator, at least one program block corresponding to the kth storage unit is mapped to the kth processing unit.

[0010] For example, in the compiler optimization method provided by at least one embodiment of the present disclosure, the parameter interface of the first conversion function includes parameter interfaces of multiple target parameters, and the multiple target parameters are used to support the second storage space access mode. Based on the first conversion function, modifying the calling method of the first operator in the compiler to convert the first operator into the second operator also includes: based on the parameter interface of the first conversion function, adding the parameter interfaces of the multiple target parameters to the operator interface of the first operator to obtain the operator interface of the second operator, wherein the operator interface of the second operator includes the parameter interface of the multiple target parameters.

[0011] For example, the compiler optimization method provided by at least one embodiment of the present disclosure also includes: calling the second operator in the compiler, wherein calling the second operator in the compiler includes: passing the multiple target parameters into the operator interface of the second operator, wherein the multiple target parameters correspond one-to-one to the parameter interfaces of the multiple target parameters.

[0012] For example, in the compiler optimization method provided in at least one embodiment of the present disclosure, the first storage space access mode is a uniform memory access mode, and the second storage space access mode is a non-uniform memory access mode.

[0013] At least one embodiment of the present disclosure further provides an electronic device, comprising: a compiler, configured to compile input data to obtain compiled data, wherein the compiled data includes a first operator; an electronic device, configured to use a first conversion function to configure the first operator, wherein the first conversion function is used to convert a first storage space access mode supported by the first operator into a second storage space access mode; the electronic device is also configured to, based on the first conversion function, modify a calling method of the first operator in the compiler to convert the first operator into a second operator, wherein the second operator supports the second storage space access mode.

[0014] At least one embodiment of the present disclosure further provides an electronic device, comprising: a processor; a memory, comprising one or more computer program modules; wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules are used to implement the compiler optimization method provided in any embodiment of the present disclosure.

[0015] At least one embodiment of the present disclosure further provides a storage medium storing non-transitory computer-readable instructions, which, when executed by a computer, implements the compiler optimization method provided by any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description only relate to some embodiments of the present disclosure, but are not intended to limit the present disclosure.

[0017] Figure 1A A schematic diagram of a unified memory access (UMA) type data access mode;

[0018] Figure 1B A schematic diagram of a non-uniform memory access (NUMA) type of data access mode;

[0019] Figure 2 An exemplary flow chart of a compiler optimization method provided for at least one embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of an example of a compiler optimization method provided by at least one embodiment of the present disclosure;

[0021] Figure 4 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure;

[0022] Figure 5 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure;

[0023] Figure 6 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure; and

[0024] Figure 7 A schematic diagram of a storage medium provided for at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution of the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0026] Unless otherwise defined, the technical terms or scientific terms used in the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, similar words such as "one", "one" or "the" do not indicate quantity restrictions, but indicate that there is at least one. Similar words such as "include" or "comprise" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Similar words such as "connect" or "connected" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0027] The present disclosure is described below through several specific embodiments. In order to keep the following description of the embodiments of the present disclosure clear and concise, detailed descriptions of known functions and known components may be omitted. When any component of the embodiments of the present disclosure appears in more than one figure, the component is represented by the same or similar reference numeral in each figure.

[0028] Deep learning computing tasks are composed of multiple operators (Operators, Ops). In artificial intelligence models, operators correspond to the computing logic of layers or nodes. For example, the convolution computing task in the convolution layer can be an operator; the weight sum computing task in the fully-connected layer (FC layer) can also be an operator. When implementing a neural network, operators can be implemented in software (for example, through a computer program) or in hardware (for example, through a circuit).

[0029] During the training or reasoning process of an AI model, multiple operators in the AI ​​model need to be called to perform predetermined computing tasks. For example, traditional AI frameworks use manual methods to optimize operators, and then establish runtime graph interpreters to solve problems such as memory allocation scheduling, while AI compilers can automatically generate operators to complete computing tasks during the optimization process. For example, specifically, using an AI compiler for deep learning compilation and optimization is the process of translating and optimizing the current deep learning computing task through one or more layers of intermediate representation (IR), and finally converting it into an executable file on the target hardware. By using an AI compiler, the efficiency of software and hardware collaboration can be improved, further meeting the requirements for hardware performance in actual deployment of computing tasks or training scenarios.

[0030] For example, in some emerging deep learning frameworks (e.g., PyTorch 2.0 framework, etc.), high-performance compilers (e.g., Inductor compiler, etc.) can be used to generate high-performance code for the backend of the accelerator, and high-performance programming languages ​​(e.g., Triton language, etc.) can be used to improve the versatility of the design to achieve high-performance computing. For example, the operator generated by the compiler can be run by calling the processing unit (or processing core) in the artificial intelligence chip, and the processing unit needs to access the corresponding storage space to read and store the tensor data of the operator. For example, the processing unit in the artificial intelligence chip can be implemented as a programmable multiprocessor, etc.

[0031] For example, some high-performance compilers only support uniform memory access (UMA) type data access mode, that is, accessing storage space through UMA type address pointers; high-performance compilers can use high-performance programming languages ​​to generate specific address pointer operators (for example, Triton operators, etc.), the input and output of which are UMA type address pointers; the compiler backend then compiles the UMA type address pointer operator into chip instruction code.

[0032] However, most AI chips also include other types of data access modes such as non-uniform memory access (NUMA). Since high-performance compilers only support UMA mode, other types of data access modes cannot be used, which greatly reduces the efficiency of data processing and thus reduces chip performance.

[0033] Figure 1A A schematic diagram of a unified memory access (UMA) type data access mode. Figure 1B A schematic diagram of a non-uniform memory access (NUMA) type of data access pattern.

[0034] For example, Figure 1A As shown, the chip includes multiple processing units (processing unit 0, processing unit 1 ... processing unit N, where N is a positive integer); in the unified memory access (UMA) mode, multiple processing units access the same storage space (for example, memory, etc.) through an interconnection bus, and all processing units have the same memory access time to access the storage space, that is, all processing units share the same memory address space.

[0035] For example, as the number of processing units in a chip increases, the UMA mode accesses storage space through a shared bus, which results in memory access bottlenecks under high loads, such as challenges to the bandwidth of the interconnect bus and conflicts in accessing the same memory block. To solve the above problems, the industry has proposed a NUMA-type data access mode.

[0036] For example, Figure 1B As shown, in the non-uniform memory access (NUMA) mode, each processing unit of the chip is directly connected to the corresponding storage unit (storage unit 0, storage unit 1... or storage unit N) through the corresponding interconnection device (interconnection 0, interconnection 1... or interconnection N), and each processing unit can directly and quickly access its own storage unit; if it needs to access the storage unit of other processing units (for example, remote memory), it needs to access it through a slower interconnection bus (not shown in the figure).

[0037] For example, compared with the UMA mode, the NUMA mode has better scalability and high performance of local access, and is more suitable for large-scale multi-processor systems, such as large database servers, high-performance computing clusters, etc., especially the design of highly integrated system-on-chip (SoC).

[0038] For example, since the high-performance compiler only supports UMA mode, in some solutions, in order to use the high-performance compiler in data access modes such as NUMA, before calling the address pointer operator, the conversion operator can be used to convert the format of the tensor data read from the storage space to UMA mode. This solution introduces additional tensor format conversion, increases the number of memory accesses to the storage space, and the call of the conversion operator increases the number of operator calls, which in turn reduces the execution efficiency of the computing task and reduces the performance of the artificial intelligence chip.

[0039] At least one embodiment of the present disclosure provides a compiler optimization method, which includes: using a compiler to compile input data to obtain compiled data, wherein the compiled data includes a first operator; using a first conversion function to configure the first operator, wherein the first conversion function is used to convert a first storage space access mode supported by the first operator into a second storage space access mode; based on the first conversion function, modifying the calling method of the first operator in the compiler to convert the first operator into a second operator, wherein the second operator supports the second storage space access mode.

[0040] At least one embodiment of the present disclosure also provides at least one electronic device and a storage medium for implementing the compiler optimization method of the above embodiment.

[0041] The method, apparatus, electronic device, and storage medium provided by at least one embodiment of the present disclosure can convert a first operator that supports a first storage space access mode in the compiler into a second operator that supports a second storage space access mode by adding a first conversion function in the first operator and modifying the calling method of the first operator in the compiler, thereby expanding the types of data access modes supported by high-performance compilers in deep learning frameworks, and eliminating the need to call conversion operators for additional tensor format conversions, thereby improving the execution efficiency of computing tasks and improving the performance of artificial intelligence chips.

[0042] Hereinafter, at least one embodiment of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that the same reference numerals in different drawings will be used to refer to the same elements described.

[0043] Figure 2 An exemplary flow chart of a compiler optimization method provided for at least one embodiment of the present disclosure.

[0044] For example, Figure 2 As shown, at least one embodiment of the present disclosure provides a compiler optimization method, which may include the following steps S110-S130.

[0045] Step S110: using a compiler to compile input data to obtain compiled data;

[0046] Step S120: configuring a first operator using a first conversion function;

[0047] Step S130: Based on the first conversion function, modify the calling method of the first operator in the compiler to convert the first operator into a second operator.

[0048] For example, in step S110, the compiled data includes a first operator. For example, the input data may be a computation graph including multiple operators. After being compiled in a compiler using a high-performance programming language, a compiled computation graph including a first operator may be obtained. The first operator may be an operator generated immediately after being compiled by the compiler (for example, a Triton operator, etc.). For example, the compiler and the first operator only support the first storage space access mode.

[0049] For example, in step S120, the first conversion function is used to convert the first storage space access mode supported by the first operator into the second storage space access mode. For example, the first conversion function can be added to the first operator to implement the conversion of the storage space access mode.

[0050] For example, in step S130, the converted second operator supports the second storage space access mode. For example, by modifying the calling method of the first operator in the compiler, the compiler and the converted second operator both support the second storage space access mode, and a mapping relationship between multiple storage units in the storage space and multiple processing units corresponding to the second storage space access mode can be implemented at the hardware level.

[0051] For example, in some examples, the first storage space access mode may be a unified memory access mode (UMA mode), and the second storage space access mode may be a non-uniform memory access mode (NUMA mode); the first storage space access mode and the second storage space access mode may also be selected as other types of storage space access modes according to actual application scenarios, and the embodiments of the present disclosure do not limit this. For example, the storage space may be a memory space, cache space, etc. inside or outside the chip, which may be selected according to actual needs, and the embodiments of the present disclosure do not limit this.

[0052] It should be noted that the compiler can be a high-performance compiler (for example, the Inductor compiler, etc.) used in a deep learning framework (for example, the PyTorch 2.0 framework, etc.), or other types of compilers can be selected according to different application scenarios or AI frameworks. The specific selection can be made according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0053] It should be noted that the first operator or the second operator can be a specific address pointer operator (for example, the Triton operator, etc.) generated by a high-performance compiler using a high-performance programming language; corresponding to different types of deep learning computing models or different execution processes of the same computing model, the first operator or the second operator can also be other types, which can be selected according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0054] It should be noted that the input data may be a FX calculation graph (Function Annotation and Transformation Graph), etc., and the high-performance programming language may be a Triton language, etc. The embodiments of the present disclosure do not limit the types of input data, high-performance programming languages, etc.

[0055] In some examples, the target storage space to be accessed includes multiple storage units, and the target storage space includes an offset address, and the offset address is used to determine the target storage unit to be accessed among the multiple storage units. For example, the target storage space can be a memory space, a cache space, etc. to be accessed, and the storage unit can be a memory block, a cache block, etc. inside the target storage space, which can be selected according to actual needs, and the embodiments of the present disclosure are not limited to this.

[0056] For example, in order to access a target storage unit in a target storage space, the location of the target storage unit can be determined by an offset address (offsets) to access it. For example, for the target storage space, the offset address used to determine the target storage unit in the first storage space access mode is different from the offset address used to determine the target storage unit in the second storage space access mode; by modifying the offset address of the target storage space, the conversion of the storage space access mode can be preliminarily achieved.

[0057] For example, Figure 2 The step S120 may further include the following steps S121~S122.

[0058] Step S121: setting a first conversion function based on the second storage space access mode;

[0059] Step S122: adding a first conversion function to the first operator.

[0060] For example, in step S121, the first conversion function is used to modify the offset address of the target storage space corresponding to the first storage space access mode to the offset address corresponding to the second storage space access mode. For example, in order to convert the first storage space access mode to the second storage space access mode, the first conversion function can be set to modify the offset address of the target storage space to access the target storage unit determined by the offset address in the target storage space based on the second storage space access mode.

[0061] For example, in step S122, by adding a first conversion function to the first operator, the first conversion function can be called in the compiler to implement conversion from the first storage space access mode to the second storage space access mode.

[0062] In some examples, the first operator includes a first address pointer for accessing the target storage space, and the first address pointer corresponds to the first storage space access mode. Figure 2 The step S130 in may further include the following step S131.

[0063] Step S131: calling a first conversion function in the first operator in the compiler to modify a first address pointer in the first operator to a second address pointer in the second operator.

[0064] For example, in step S131, the second address pointer corresponds to the second storage space access mode. For example, the first address pointer is calculated from the offset address corresponding to the first storage space access mode, and the second address pointer is calculated from the offset address corresponding to the second storage space access mode.

[0065] For example, the first address pointer or the second address pointer for accessing the target storage space can be calculated in the process of loading and storing. For example, specifically, taking the loading process as an example, assuming that the offset address x0 corresponding to the first storage space access mode is used as the original input, in the original first operator, the calculation process of the first address pointer tmp0 can be simply illustrated by the following pseudo code:

[0066] tmp0 = tl.load(x0);

[0067] For example, in the second operator obtained by conversion, the calculation process of the second address pointer tmp0' can be simply illustrated by the following pseudo code:

[0068] tmp0' = tl.load(tl.supa_offset(x0, para0, para1…)),

[0069] Wherein, tl.supa_offset() represents the first conversion function called in the second operator, and para0, para1... represent multiple parameter types that need to be input in the function interface of the first conversion function. For example, by calling the first conversion function, the offset address corresponding to the second storage space access mode is modified to tl.supa_offset(x0, para0, para1...).

[0070] For example, because the first conversion function is called in the compiler when calculating the second address pointer, the first address pointer corresponding to the first storage space access mode is modified to the second address pointer corresponding to the second storage space access mode.

[0071] In some examples, the target storage space further includes a first address, which is used to determine a mapping relationship between the plurality of storage units and the plurality of processing units, and the plurality of processing units are used to access the target storage space. Figure 2 The step S130 in may further include the following step S132.

[0072] Step S132: modifying the first address of the target storage space defined in the first operator to the first address of the target storage space defined in the second operator in the compiler, so as to modify the mapping relationship between the multiple storage units and the multiple processing units.

[0073] For example, in steps S121-S122, the offset address of the target storage space has been modified based on the first conversion function, and the conversion of the first storage space access mode into the second storage space access mode has been preliminarily realized; based on the modification of the first address (or offset) of the target storage space in step S132, the mapping relationship between the multiple storage units in the target storage space and the multiple processing units corresponding to the second storage space access mode can be further realized at the hardware level. For example, in step S132, the modified mapping relationship between the multiple storage units and the multiple processing units supports the second storage space access mode.

[0074] For example, the plurality of processing units include a first processing unit to an Nth processing unit, and the plurality of storage units include a first storage unit to an Nth storage unit, where N is an integer greater than 1. For example, in the second storage space access mode, the kth processing unit of the plurality of processing units is configured to access the kth storage unit of the plurality of storage units, where k= 1, 2,…, N.

[0075] For example, specifically, each storage unit in the plurality of storage units corresponds to at least one program block; in the mapping relationship determined by the first address defined in the second operator, at least one program block corresponding to the kth storage unit is mapped to the kth processing unit.

[0076] For example, in order to adjust the mapping relationship determined by the first address, in step S132, the first address of the target storage space defined in the first operator is modified to the first address of the target storage space defined in the second operator. For example, the first address of the target storage space defined in the first operator is represented as Xoffset, and the space size corresponding to the storage unit is represented as XBlock. By calling the tl.program_id(0) function in the first operator (tl.program_id(0) indicates the number of the storage unit), the calculation process of the first address Xoffset can be simply illustrated by the following pseudo code:

[0077] Xoffset = tl.program_id(0) × XBlock;

[0078] For example, by calling the tl.program_id(0) function and the tl.cdiv() function in the second operator, the calculation process of the first address Xoffset' of the target storage space defined in the second operator can be simply illustrated by the following pseudo code:

[0079] pid = tl.program_id(0)

[0080] nb = xnumel / / XBlock

[0081] pid = (pid % N) × tl. cdiv(nb, N) + pid / / N

[0082] Xoffset' = pid × XBlock.

[0083] For example, nb represents the number of storage units, xnumel represents the number of data elements in the storage unit, and XBlock represents the size of the space corresponding to the storage unit. Dividing xnumel by XBlock can get the number of storage units nb; pid and tl.program_id(0) represent the number of storage units, and tl.cdiv() represents a division function that rounds up. Through the above calculation process, the starting address Xoffset' of the target storage space can be obtained.

[0084] For example, in one example, assume that the number of processing units and storage units is 16 (ie, N=16), each storage unit corresponds to 7 program blocks, and the total number of program blocks is 16 × 7 = 112; the 1st to 7th program blocks correspond to the 1st storage unit, the 8th to 14th program blocks correspond to the 2nd storage unit... and the 106th to 112th program blocks correspond to the 16th storage unit.

[0085] For example, assume that before the mapping relationship is modified, 16 processing units access 16 storage units in a polling manner, and the specific polling manner is: allocate the 1st program block to the 1st processing unit, allocate the 2nd program block to the 2nd processing unit... allocate the 112th program block to the 16th processing unit. For example, in this case, although the conversion of the first storage space access mode (e.g., UMA mode) to the second storage space access mode (e.g., NUMA mode) has been implemented based on the first conversion function, the one-to-one access relationship required by the second storage space access mode may not be satisfied between the multiple storage units and the multiple processing units, so it is also necessary to adjust the mapping relationship between the multiple storage units and the multiple processing units at the hardware level.

[0086] For example, by modifying the first address of the target storage space in step S132, the way in which 16 processing units access 16 storage units is changed from the polling method in the above example to a sequential method, and the specific sequential method is: assigning the 1st to 7th program blocks to the 1st processing unit, assigning the 8th to 14th program blocks to the 2nd processing unit... assigning the 106th to 112th program blocks to the 16th processing unit. For example, in this case, the one-to-one corresponding access relationship required by the second storage space access mode is satisfied between the multiple storage units and the multiple processing units, thereby optimizing the control function of the compiler at the hardware level and further expanding the types of data access modes supported by the compiler.

[0087] It should be noted that if the mapping relationship between multiple storage units and multiple processing units is itself sequential, step S132 can be omitted and the current mapping relationship can be used directly; the mapping relationship between multiple storage units and multiple processing units is not limited to the polling method and sequential method described above, and can be specifically selected according to the actual application scenario, and the embodiments of the present disclosure do not limit this.

[0088] In some examples, the parameter interface of the first conversion function includes parameter interfaces of multiple target parameters, and the multiple target parameters are used to support the second storage space access mode. Figure 2 The step S130 in may further include the following step S133.

[0089] Step S133: Based on the parameter interface of the first conversion function, the parameter interfaces of multiple target parameters are added to the operator interface of the first operator to obtain the operator interface of the second operator.

[0090] For example, in step S133, the operator interface of the original first operator only includes an input data interface and an output data interface, which can support the first storage space access mode; in order to support the second storage space access mode, the parameter interface of the first conversion function, in addition to the input data interface and the output data interface, also needs to include parameter interfaces of multiple target parameters related to information such as operator tensor data.

[0091] For example, taking the first conversion function supa_offset(x0, para0, para1...) as an example, the parameter interface of the first conversion function supa_offset() includes not only the offset address x0, but also parameter interfaces para0, para1... of multiple target parameters related to information of operator tensor data. For example, taking tensor data with a data shape of [N, C, H, W] as an example, multiple target parameters may include a batch N, the number of channels C, the height H, the width W, and other required parameters of the tensor data; the types and quantities of the multiple target parameters may be selected according to actual needs, and the embodiments of the present disclosure are not limited thereto.

[0092] For example, in step S133, in the process of converting the first operator to the second operator, it is necessary to add parameter interfaces of multiple target parameters to the operator interface of the first operator so that the operator interface of the converted second operator contains parameter interfaces of multiple target parameters to be compatible with the first conversion function, thereby supporting the second storage space access mode.

[0093] In some examples, the compiler optimization method provided by at least one embodiment of the present disclosure further includes step S140: calling the second operator in the compiler.

[0094] For example, during the execution of step S140, step S133 may further include: passing multiple target parameters into the operator interface of the second operator. For example, the multiple target parameters correspond one-to-one to the parameter interfaces of the multiple target parameters.

[0095] For example, in the actual calling process of the second operator, it is necessary to run corresponding code in the compiler to pass in the actual target parameter corresponding to the operator interface of the second operator, so as to use the target parameter to perform calculations on the input tensor data in the second operator. For example, the actual target parameter can be a parameter related to the tensor data input to the second operator, etc., which is not limited in the embodiments of the present disclosure.

[0096] Figure 3 A schematic diagram of an example of a compiler optimization method provided by at least one embodiment of the present disclosure. For example, Figure 3 for Figure 2 A specific example of the compiler's optimization method is shown.

[0097] For example, Figure 3 As shown, the target storage space 100 to be accessed includes N storage units, where N is a positive integer; the target storage space 100 includes an offset address and a first address, the offset address is used to determine the target storage unit to be accessed among the N storage units, and the first address is used to determine the mapping relationship between the N storage units and N processing units (not shown in the figure).

[0098] For example, Figure 3 As shown, the offset address corresponds to storage unit i (here i = 1, 2, ..., N), so the storage unit i is determined as the target storage unit to be accessed. Figure 2 The first operator obtained in step S110 is Figure 2 In step S120, the first operator is configured using the first conversion function. For example, specifically, the first conversion function is used to modify the offset address of the target storage space 100 corresponding to the first storage space access mode to the offset address corresponding to the second storage space access mode.

[0099] For example, Figure 3 As shown, the first address corresponds to the storage unit (eg, storage unit 0) at the first position in the target storage space 100. Figure 2 In step S130, based on the first conversion function, the calling mode of the first operator in the compiler is modified to convert the first operator into the second operator. For example, specifically, step S131 is executed to call the first conversion function in the compiler to modify the first address pointer in the first operator to the second address pointer in the second operator; in addition, step S133 is executed to add parameter interfaces of multiple target parameters to the operator interface of the first operator based on the parameter interface of the first conversion function to obtain the operator interface of the second operator.

[0100] For example, optionally, when the one-to-one access relationship required by the second storage space access mode (for example, the NUMA mode) is not satisfied between the N storage units and the N processing units, step S132 is executed to modify the starting address of the target storage space 100 defined in the first operator to the starting address of the target storage space 100 defined in the second operator in the compiler to modify the mapping relationship between multiple storage units and multiple processing units.

[0101] It should be noted that Figure 3 The offset address, first address, target storage space, storage unit and corresponding relationship shown are only examples. The embodiments of the present disclosure do not limit the specific implementation methods of the type and size of the target storage space, the types and numbers of storage units and processing units, the offset address and the first address, etc.

[0102] In the compiler optimization method provided by at least one embodiment of the present disclosure, Figure 2 Steps S110 to S130 are a process of converting the function definition of the first operator into the function definition of the second operator, and step S140 is a process of calling the function of the second operator new_op.

[0103] For example, in some examples, the first operator is represented as old_op, and the second operator is represented as new_op. In the compiler, the process of converting the function definition of the first operator old_op into the function definition of the second operator new_op can be simply illustrated by the following pseudo code:

[0104] def old_op(float×inputs, float×outputs):

[0105] Xoffset = tl.program_id(0) × XBlock

[0106] tmp0 = tl.load(x0)

[0107] -->

[0108] def new_op(float×inputs, float×outputs, para0, para1…):

[0109] pid = tl.program_id(0)

[0110] nb = xnumel / / XBlock

[0111] pid = (pid % N) × tl. cdiv(nb, N) + pid / / N

[0112] Xoffset' = pid × XBlock

[0113] tmp0' = tl.load(tl.supa_offset(x0, para0, para1…))

[0114] Wherein, float×inputs represents the input data interface of the first operator or the second operator, float×outputs represents the output data interface of the first operator or the second operator, para0, para1… represent parameter interfaces of multiple target parameters, x0 represents the offset address, and N represents the number of storage units. For example, the specific explanation of other parameters and statements in the above pseudo code can refer to the description above, which will not be repeated here.

[0115] For example, in a compiler, the process of converting a function call of a first operator old_op into a function call of a second operator new_op can be simply illustrated by the following pseudo code:

[0116] old_op(inputs)

[0117] -->

[0118] new_op(inputs, para0, para1…)

[0119] Among them, inputs represents the input data actually passed in the operator interface of the first operator or the second operator, and para0, para1... represent multiple target parameters actually passed in the operator interface of the first operator or the second operator. For example, the specific explanation of other parameters and statements in the above pseudo code can refer to the description above, which will not be repeated here.

[0120] It should be noted that the pseudo codes provided in at least one embodiment of the present disclosure are all illustrative, and the specific codes need to be set according to actual needs, and the embodiments of the present disclosure are not limited to this. For example, the specific implementation of the code can refer to the description in the art, which will not be repeated here.

[0121] Figure 4 A schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0122] For example, Figure 4 As shown, the electronic device 200 includes a compiler 210 and an electronic device 220 .

[0123] For example, the compiler 210 is configured to compile input data to obtain compiled data, wherein the compiled data includes the first operator. That is, the compiler 210 can be configured to execute, for example Figure 2 Step S110 is shown.

[0124] For example, the electronic device 220 is configured to configure the first operator using a first conversion function. For example, the first conversion function is used to convert a first storage space access mode supported by the first operator into a second storage space access mode. That is, the electronic device 220 can be configured to perform, for example Figure 2 Step S120 is shown.

[0125] For example, the electronic device 220 is further configured to modify the calling method of the first operator in the compiler based on the first conversion function to convert the first operator into a second operator. For example, the second operator supports the second storage space access mode. That is, the electronic device 220 can also be configured to perform, for example Figure 2 Step S130 is shown.

[0126] In some examples, the first storage space access mode is a uniform memory access mode, and the second storage space access mode is a non-uniform memory access mode.

[0127] In some examples, the target storage space to be accessed includes a plurality of storage units, the target storage space includes an offset address, and the offset address is used to determine the target storage unit to be accessed in the plurality of storage units. For example, the electronic device 220 is further configured to: set a first conversion function based on the second storage space access mode; and add the first conversion function to the first operator. For example, the first conversion function is used to modify the offset address of the target storage space corresponding to the first storage space access mode to the offset address corresponding to the second storage space access mode.

[0128] In some examples, the first operator includes a first address pointer for accessing the target storage space, and the first address pointer corresponds to a first storage space access mode. For example, the electronic device 220 is further configured to call a first conversion function in the first operator in a compiler to modify the first address pointer in the first operator to a second address pointer in the second operator. For example, the second address pointer corresponds to a second storage space access mode.

[0129] For example, the target storage space includes a plurality of storage units, the target storage space includes an offset address, and the offset address is used to determine the target storage unit to be accessed among the plurality of storage units. For example, the first address pointer is calculated from the offset address corresponding to the first storage space access mode, and the second address pointer is calculated from the offset address corresponding to the second storage space access mode.

[0130] In some examples, the target storage space to be accessed includes multiple storage units, the target storage space includes a first address, the first address is used to determine the mapping relationship between the multiple storage units and the multiple processing units, and the multiple processing units are used to access the target storage space. For example, the electronic device 220 is also configured to modify the first address of the target storage space defined in the first operator to the first address of the target storage space defined in the second operator in the compiler to modify the mapping relationship between the multiple storage units and the multiple processing units. For example, the modified mapping relationship between the multiple storage units and the multiple processing units supports the second storage space access mode.

[0131] For example, the plurality of processing units include a first processing unit to an Nth processing unit, and the plurality of storage units include a first storage unit to an Nth storage unit, where N is an integer greater than 1. For example, in the second storage space access mode, the kth processing unit is configured to access the kth storage unit, where k = 1, 2, ..., N.

[0132] For example, each storage unit in the plurality of storage units corresponds to at least one program block. For example, in the mapping relationship determined by the first address defined in the second operator, at least one program block corresponding to the kth storage unit is mapped to the kth processing unit.

[0133] In some examples, the parameter interface of the first conversion function includes parameter interfaces of multiple target parameters, and the multiple target parameters are used to support the second storage space access mode. For example, the electronic device 220 is further configured to add parameter interfaces of multiple target parameters to the operator interface of the first operator based on the parameter interface of the first conversion function to obtain the operator interface of the second operator. For example, the operator interface of the second operator includes parameter interfaces of multiple target parameters.

[0134] In some examples, the electronic device 220 is further configured to call the second operator in the compiler. For example, the electronic device 220 is further configured to pass multiple target parameters to the operator interface of the second operator. For example, the multiple target parameters correspond one-to-one to the parameter interfaces of the multiple target parameters.

[0135] Since in the above description, for example Figure 2 In the process of the compiler optimization method shown in FIG. 1 , the details of the operation of the electronic device 200 have been introduced, so for the sake of brevity, they will not be repeated here. For relevant details, please refer to the above Figure 2~Figure 3 etc.

[0136] It should be noted that Figure 4 The electronic device 220 in the electronic device 200 shown may include multiple functional modules, and each module may be configured as software, hardware, firmware, or any combination of the above items to perform a specific function. For example, these modules may correspond to dedicated integrated circuits, pure software codes, or modules that combine software and hardware. In addition, it is clear to those skilled in the art that the processing performed by each module may also be performed in the electronic device 220 without any specific module division or without clear demarcation between the modules. As an example, refer to Figure 4 The electronic device 200 described may be a PC computer, a tablet device, a personal digital assistant, a smart phone, a web application or other device capable of executing program instructions, but is not limited thereto.

[0137] At least one embodiment of the present disclosure also provides another electronic device, which includes a processor and a memory; the memory includes one or more computer program modules; the one or more computer program modules are stored in the memory and are configured to be executed by the processor, and the one or more computer program modules include a compiler optimization method for implementing the above-described embodiment of the present disclosure. For example, the processor can be a single-core processor or a multi-core processor.

[0138] Figure 5 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0139] For example, Figure 5As shown, the electronic device 300 includes a processor 310 and a memory 320. For example, the memory 320 is used to store non-transitory computer-readable instructions (e.g., one or more computer program modules). The processor 310 is used to run non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are run by the processor 310, one or more steps of the optimization method of the compiler described above can be executed. The memory 320 and the processor 310 can be interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0140] For example, the processor 310 may be a central processing unit (CPU), a graphics processing unit (GPU), a general purpose graphics processing unit (GPGPU), a digital signal processor (DSP), or other forms of processing units, such as a field programmable gate array (FPGA), etc.; for example, the central processing unit (CPU) may be an X86, RISC-V, or ARM architecture, etc. The processor 310 may be a general-purpose processor or a dedicated processor, and may control other components in the electronic device 300 to perform desired functions.

[0141] For example, the memory 320 may include any combination of one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 310 may run one or more computer program modules to implement various functions of the electronic device 300. Various applications and various data, as well as various data used and / or generated by the application, etc. may also be stored in the computer-readable storage medium.

[0142] It should be noted that, in the embodiments of the present disclosure, the specific functions and technical effects of the electronic device 300 can refer to the above description of the compiler optimization method provided in at least one embodiment of the present disclosure, and will not be repeated here.

[0143] Figure 6 A schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0144] For example, Figure 6 As shown, the electronic device 400 is suitable for implementing the compiler optimization method provided by the embodiment of the present disclosure. It should be noted that, Figure 6The electronic device 400 shown is only an example and does not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0145] For example, Figure 6 As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 41, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 42 or a program loaded from a storage device 48 to a random access memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 41, the ROM 42, and the RAM 43 are connected to each other through a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44. Generally, the following devices can be connected to the I / O interface 45: an input device 46 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 47 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 48 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 49. The communication device 49 can allow the electronic device 400 to communicate with other electronic devices wirelessly or by wire to exchange data.

[0146] Although Figure 6 The electronic device 400 is shown to have various devices, but it should be understood that it is not required to implement or possess all the devices shown, and the electronic device 400 may alternatively implement or possess more or fewer devices.

[0147] For the detailed description and technical effects of the electronic device 400, please refer to the above description of the compiler optimization method, which will not be repeated here.

[0148] Figure 7 A schematic diagram of a storage medium provided for at least one embodiment of the present disclosure.

[0149] For example, Figure 7 As shown, the storage medium 500 stores non-transitory computer-readable instructions 510. For example, when the non-transitory computer-readable instructions 510 are executed by a computer, one or more steps in the optimization method of the compiler described above are performed.

[0150] For example, the storage medium 500 can be applied to Figure 5 For example, the storage medium 500 may be the memory 320 in the electronic device 300. For example, the description of the storage medium 500 may refer to Figure 5 The corresponding description of the memory 320 in the electronic device 300 is shown and will not be repeated here.

[0151] There are a few points to note about this disclosure:

[0152] (1) In the drawings of the embodiments of the present disclosure, only the structures related to the embodiments of the present disclosure are involved, and other structures can refer to the general design.

[0153] (2) Unless there is a conflict, the features of the same embodiment or different embodiments of the present disclosure may be combined with each other.

[0154] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A compiler optimization method, characterized in that: The optimization method of the compiler includes: Using the compiler to compile input data to obtain compiled data, wherein the compiled data includes a first operator; configuring the first operator using a first conversion function, wherein the first conversion function is used to convert a first storage space access mode supported by the first operator into a second storage space access mode; Based on the first conversion function, modify the calling mode of the first operator in the compiler to convert the first operator into a second operator, wherein the second operator supports the second storage space access mode, The first storage space access mode is a uniform memory access mode, and the second storage space access mode is a non-uniform memory access mode. The target storage space to be accessed includes a plurality of storage units, and the target storage space includes an offset address, and the offset address is used to determine the target storage unit to be accessed among the plurality of storage units. The configuring the first operator using the first conversion function includes: Setting the first conversion function based on the second storage space access mode, wherein the first conversion function is used to modify the offset address of the target storage space corresponding to the first storage space access mode to an offset address corresponding to the second storage space access mode; The first conversion function is added to the first operator.

2. The compiler optimization method according to claim 1, characterized in that: The first operator includes a first address pointer for accessing the target storage space, the first address pointer corresponds to the first storage space access mode, The modifying, based on the first conversion function, the calling mode of the first operator in the compiler to convert the first operator into a second operator includes: A first conversion function in the first operator is called in the compiler to modify a first address pointer in the first operator to a second address pointer in the second operator, wherein the second address pointer corresponds to the second storage space access mode.

3. The compiler optimization method according to claim 2, characterized in that: The target storage space includes a plurality of storage units, the target storage space includes the offset address, and the offset address is used to determine a target storage unit to be accessed among the plurality of storage units. The first address pointer is calculated from an offset address corresponding to the first storage space access mode, and the second address pointer is calculated from an offset address corresponding to the second storage space access mode.

4. The compiler optimization method according to claim 1, characterized in that: The target storage space includes a plurality of storage units, the target storage space includes a first address, the first address is used to determine a mapping relationship between the plurality of storage units and a plurality of processing units, the plurality of processing units are used to access the target storage space, The modifying the calling mode of the first operator in the compiler to convert the first operator into a second operator includes: In the compiler, the first address of the target storage space defined in the first operator is modified to the first address of the target storage space defined in the second operator, so as to modify the mapping relationship between the multiple storage units and the multiple processing units. The modified mapping relationship between the multiple storage units and the multiple processing units supports the second storage space access mode.

5. The compiler optimization method according to claim 4, characterized in that: The plurality of processing units include a first processing unit to an Nth processing unit, and the plurality of storage units include a first storage unit to an Nth storage unit, wherein N is an integer greater than 1, In the second storage space access mode, a kth processing unit among the plurality of processing units is configured to access a kth storage unit among the plurality of storage units, where k=1, 2, ..., N.

6. The compiler optimization method according to claim 5, characterized in that: Each of the plurality of storage units corresponds to at least one program block. In the mapping relationship determined by the first address defined in the second operator, at least one program block corresponding to the k-th storage unit is mapped to the k-th processing unit.

7. The compiler optimization method according to claim 1, characterized in that: The parameter interface of the first conversion function includes parameter interfaces of multiple target parameters, and the multiple target parameters are used to support the second storage space access mode. The modifying, based on the first conversion function, the calling mode of the first operator in the compiler to convert the first operator into a second operator includes: Based on the parameter interface of the first conversion function, the parameter interfaces of the multiple target parameters are added to the operator interface of the first operator to obtain the operator interface of the second operator, wherein the operator interface of the second operator includes the parameter interfaces of the multiple target parameters.

8. The compiler optimization method according to claim 7, characterized in that: The optimization method of the compiler also includes: calling the second operator in the compiler, The calling of the second operator in the compiler includes: The multiple target parameters are input into the operator interface of the second operator, wherein the multiple target parameters correspond one-to-one to the parameter interfaces of the multiple target parameters.

9. An electronic device, characterized in that: The electronic device comprises: A compiler is configured to compile input data to obtain compiled data, wherein the compiled data includes a first operator; an electronic device configured to configure the first operator using a first conversion function, wherein the first conversion function is used to convert a first storage space access mode supported by the first operator into a second storage space access mode; The electronic device is further configured to modify, based on the first conversion function, a calling mode of the first operator in the compiler to convert the first operator into a second operator, wherein the second operator supports the second storage space access mode, The first storage space access mode is a uniform memory access mode, and the second storage space access mode is a non-uniform memory access mode. The target storage space to be accessed includes a plurality of storage units, the target storage space includes an offset address, and the offset address is used to determine the target storage unit to be accessed among the plurality of storage units, and the electronic device is further configured as follows: Setting the first conversion function based on the second storage space access mode, wherein the first conversion function is used to modify the offset address of the target storage space corresponding to the first storage space access mode to an offset address corresponding to the second storage space access mode; The first conversion function is added to the first operator.

10. An electronic device, characterized in that: The electronic device comprises: processor; a memory including one or more computer program modules; The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules are used to implement the compiler optimization method according to any one of claims 1 to 8.

11. A storage medium, characterized in that: The storage medium stores non-transitory computer-readable instructions, and when the non-transitory computer-readable instructions are executed by a computer, the compiler optimization method according to any one of claims 1 to 8 is implemented.

Images