A method, compiler and server for constructing intermediate representation

Abstract

An embodiment of the present application provides a method for constructing an intermediate expression, the method comprising: obtaining a first intermediate expression IR, the first IR comprising a calculation statement, the calculation statement comprising a tensor and an operator, wherein the operation represented by the operator is executed based on a calculation unit, the calculation unit is used to perform operations on data migrated through a first storage location and a second storage location in sequence, and the tensor is data used to perform the operation represented by the operator; generating a second IR based on the calculation statement, the second IR comprising first data migration information and data segmentation information, the first data migration information indicating that the first migration path of the tensor includes from the first storage location to the second storage location, and the data segmentation information indicating the size of the tensor when the tensor migrates to the second storage location. The present application can construct an intermediate expression that can express the migration of tensors between different storage locations.

Description

[0001] This application claims the priority of the Chinese patent application filed with the China Patent Office on September 18, 2019, with application number 201910896548.1 and invention name “A method, compiler and server for constructing intermediate expressions”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of electronic technology, and in particular to a method, compiler and server for constructing an intermediate expression. Background Art

[0003] With the development of artificial intelligence (AI) technology, AI chips based on domain specific architecture (DSA) are emerging. Unlike modern microprocessors, AI chips based on DSA use multiple storage locations to shorten the distance of data movement and reduce the overhead caused by data movement. With the evolution of DSA, the increase of computing units and the improvement of operator complexity, the number of storage locations has increased exponentially. At this time, it is necessary to move tensors to the corresponding computing units through multiple storage locations to realize the execution of operators.

[0004] However, in the prior art, there is no method for constructing an intermediate representation (IR) that can migrate data to a computing unit through multiple storage locations. Summary of the invention

[0005] The present application embodiment provides a method for constructing an intermediate expression, comprising:

[0006] Obtaining a first intermediate expression IR, the first IR including a calculation statement, the calculation statement including a tensor and an operator, wherein the operation represented by the operator is performed based on a calculation unit, the calculation unit is used to perform an operation on data migrated through a first storage location and a second storage location in sequence, and the tensor is data used to perform the operation represented by the operator;

[0007] A second IR is generated based on the calculation statement, and the second IR includes first data migration information and data segmentation information, wherein the first data migration information indicates that the first migration path of the tensor includes from the first storage location to the second storage location, wherein there may be multiple data transmission paths between the first storage location and the second storage location, and some data transmission paths may not have other storage locations, for example, the data transmission path between the first storage location and the second storage location described below is directly connected; some data paths may have other storage locations, such as the third storage location mentioned below. The data segmentation information indicates the size of the tensor when the tensor is migrated to the second storage location.

[0008] Optionally, in an optional design of the first aspect, the computing unit is also used to perform operations on data passing through a second migration path, the second migration path comprising from the first storage location, through at least one third storage location, to the second storage location, the first migration path and the second migration path being different data migration paths.

[0009] Optionally, in an optional design of the first aspect, the at least one third storage location indicates a storage location on an on-chip cache.

[0010] Optionally, in an optional design of the first aspect, the method further includes:

[0011] generating first data flow information corresponding to the calculation statement based on the calculation statement, the first data flow information including second data migration information of the tensor, the second data migration information indicating that the tensor is migrated from the first storage position to the second storage position through at least one third migration position in sequence;

[0012] Based on the existence of a direct data transmission path between the first storage location and the second storage location, second data flow information is generated, wherein the second data flow information includes the first data migration information.

[0013] In this embodiment, the existence of a direct data transmission path between the first storage location and the second storage location can be understood as: the existence of a data transmission path between the first storage location and the second storage location without passing through any other storage location.

[0014] For example, the tensor is used to migrate to the computing unit through a first storage location, at least one third storage location, and a second storage location in sequence. For another example, the tensor is used to migrate to the computing unit through at least one fourth storage location, the first storage location, and the second storage location in sequence. For another example, the tensor is used to migrate to the computing unit through at least one fourth storage location, the first storage location, at least one third storage location, and the second storage location in sequence. For another example, the tensor is used to migrate to the computing unit through at least one fourth storage location, the first storage location, at least one third storage location, the second storage location, and at least one fifth storage location in sequence.

[0015] Optionally, in an optional design of the first aspect, the first storage location indicates a storage location on an on-chip cache, and the second storage location indicates a storage location on the on-chip cache.

[0016] Optionally, in an optional design of the first aspect, the first storage location indicates a memory, and the second storage location indicates a storage location on an on-chip cache.

[0017] Optionally, in an optional design of the first aspect, the tensor includes multiple dimensions, each dimension corresponds to an axis variable, and the axis variable is used to represent the size of the tensor of the corresponding dimension. The data splitting information includes multiple axis variables and a splitting axis variable corresponding to each axis variable, and the splitting axis variable is used to represent the size of the tensor of the corresponding dimension when the tensor is migrated.

[0018] Optionally, in an optional design of the first aspect, the second IR also includes at least one target variable and a value range of each target variable, the axis variable is linearly related to the at least one target variable, and the axis variable represents the tensor size of the corresponding dimension through the at least one target variable and the value range of the at least one target variable.

[0019] Optionally, in an optional design of the first aspect, the second IR is an IR of a tree structure, the tree structure includes a root node and child nodes, the root node corresponds to the calculation statement, and the child node corresponds to the second storage location, wherein the second IR includes information of the child node corresponding to the second storage location, and the information of the child node corresponding to the second storage location includes the first data migration information and the data segmentation information.

[0020] Optionally, in an optional design of the first aspect, the second IR also includes: data movement information, and the data movement information indicates that the tensor is moved from the first storage location to the second storage location.

[0021] Optionally, in an optional design of the first aspect, the method further includes:

[0022] The first storage location and the second storage location are determined according to the type of the computing unit.

[0023] In a second aspect, the present application provides a compiler, comprising:

[0024] an acquisition unit, configured to acquire a first intermediate expression IR, wherein the first IR includes a calculation statement, the calculation statement includes a tensor and an operator, wherein the operation represented by the operator is performed based on the calculation unit, the calculation unit is configured to perform an operation on data migrated sequentially through the first storage location and the second storage location, and the tensor is data used to perform the operation represented by the operator;

[0025] A processing unit is used to generate a second IR based on the calculation statement, the second IR includes first data migration information and data splitting information, the first data migration information indicates that the first migration path of the tensor includes from the first storage location to the second storage location, and the data splitting information indicates the size of the tensor when the tensor is migrated to the second storage location.

[0026] Optionally, in an optional design of the second aspect, the computing unit is also used to perform operations on data passing through a second migration path, the second migration path comprising from the first storage location, through at least one third storage location, to the second storage location, the first migration path and the second migration path being different data migration paths.

[0027] Optionally, in an optional design of the second aspect, the at least one third storage location indicates a storage location on an on-chip cache.

[0028] Optionally, in an optional design of the second aspect, the processing module is further used to: generate first data flow information corresponding to the calculation statement based on the calculation statement, the first data flow information including second data migration information of the tensor, and the second data migration information indicates that the tensor is migrated from the first storage position to the second storage position through at least one third migration position in sequence;

[0029] Based on the existence of a direct data transmission path between the first storage location and the second storage location, second data flow information is generated, wherein the second data flow information includes the first data migration information.

[0030] Optionally, in an optional design of the second aspect, the first storage location indicates a storage location on an on-chip cache, and the second storage location indicates a storage location on the on-chip cache.

[0031] Optionally, in an optional design of the second aspect, the first storage location indicates a memory, and the second storage location indicates a storage location on an on-chip cache.

[0032] Optionally, in an optional design of the second aspect, the tensor includes multiple dimensions, each dimension corresponds to an axis variable, and the axis variable is used to represent the size of the tensor of the corresponding dimension. The data splitting information includes multiple axis variables and a splitting axis variable corresponding to each axis variable, and the splitting axis variable is used to represent the size of the tensor of the corresponding dimension when the tensor is migrated.

[0033] Optionally, in an optional design of the second aspect, the second IR also includes at least one target variable and a value range of each target variable, the axis variable is linearly related to the at least one target variable, and the axis variable represents the tensor size of the corresponding dimension through the at least one target variable and the value range of the at least one target variable.

[0034] Optionally, in an optional design of the second aspect, the second IR is an IR of a tree structure, the tree structure includes a root node and child nodes, the root node corresponds to the calculation statement, and the child node corresponds to the second storage location, wherein the second IR includes information of the child node corresponding to the second storage location, and the information of the child node corresponding to the second storage location includes the first data migration information and the data segmentation information.

[0035] Optionally, in an optional design of the second aspect, the second IR also includes: data movement information, and the data movement information indicates that the tensor is moved from the first storage location to the second storage location.

[0036] Optionally, in an optional design of the second aspect, the processing unit is further used for:

[0037] The first storage location and the second storage location are determined according to the type of the computing unit.

[0038] In a third aspect, the present application provides a computer system, including: a processor and a memory;

[0039] The processor is electrically connected to the memory;

[0040] The processor is used to call the code in the memory to execute any method as described in the first aspect above.

[0041] In a fourth aspect, the present application provides a computer storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes any method described in the first aspect above.

[0042] In a fifth aspect, the present application embodiment provides a method for constructing an intermediate expression, comprising:

[0043] Obtain a first intermediate expression IR, where the first IR includes a calculation statement, the calculation statement includes a tensor and an operator, the operation represented by the operator is performed based on a calculation unit, the tensor is used to migrate to the calculation unit through a first storage location and a second storage location in sequence, and the tensor is data used to perform the operation represented by the operator;

[0044] A second IR is generated based on the calculation statement, and the second IR includes first data migration information and data splitting information. The first data migration information indicates that the migration path of the tensor includes from the first storage location to the second storage location, and the data splitting information indicates the size of the tensor migrated to the second storage location each time.

[0045] Optionally, the first storage location may be a starting storage location on the tensor migration path.

[0046] Optionally, the first storage location and the second storage location may be adjacent storage locations.

[0047] Optionally, other storage locations may be spaced between the first storage location and the second storage location.

[0048] Optionally, other storage locations may be spaced between the second storage location and the computing unit.

[0049] For example, the tensor is used to migrate to the computing unit through a first storage location, at least one third storage location, and a second storage location in sequence. For another example, the tensor is used to migrate to the computing unit through at least one fourth storage location, the first storage location, and the second storage location in sequence. For another example, the tensor is used to migrate to the computing unit through at least one fourth storage location, the first storage location, at least one third storage location, and the second storage location in sequence. For another example, the tensor is used to migrate to the computing unit through at least one fourth storage location, the first storage location, at least one third storage location, the second storage location, and at least one fifth storage location in sequence.

[0050] Optionally, in an optional design of the fifth aspect, the generating a second IR based on the calculation statement includes: generating a second IR based on the calculation statement, the storage size corresponding to the first storage location, and the storage size corresponding to the second storage location, the second IR including first data migration information and data splitting information, the first data migration information indicating that the migration path of the tensor includes from the first storage location to the second storage location, and the data splitting information indicates the size of the tensor migrated to the second storage location each time.

[0051] Optionally, in an optional design of the fifth aspect, the first storage location indicates a storage location on an on-chip cache, and the second storage location indicates a storage location on the on-chip cache.

[0052] Optionally, in an optional design of the fifth aspect, the first storage location is a memory, and the second storage location indicates a storage location on an on-chip cache.

[0053] Optionally, in an optional design of the fifth aspect, the tensor corresponds to the target variable in the calculation statement, and the second IR also includes: a value range of the target variable, and a mapping of the target variable to the tensor of the first storage location.

[0054] Optionally, in an optional design of the fifth aspect, the data segmentation information includes the value range of the target variable corresponding to each time the tensor is migrated to the second storage location.

[0055] Optionally, in an optional design of the fifth aspect, the tensor includes a target axis variable, the split tensor includes a split axis variable of the target axis variable, and the data split information includes the size relationship between the split axis variable and the target axis variable each time the tensor is migrated to the second storage location.

[0056] Optionally, in an optional design of the fifth aspect, the second IR includes node information corresponding to the second cache, and the node information includes the first data migration information and the data segmentation information.

[0057] Optionally, in an optional design of the fifth aspect, the second IR also includes: read and write information, and the read and write information indicates that the second storage location reads the tensor from the first storage location.

[0058] Optionally, in an optional design of the fifth aspect, the method further includes:

[0059] Generate first data flow information corresponding to the calculation statement, the first data flow information including first data migration information associated with the tensor, the first data migration information indicating that the tensor is migrated from the first storage location to the second storage location.

[0060] Optionally, in an optional design of the fifth aspect, the generating the first data flow information corresponding to the calculation statement includes:

[0061] Generate first data flow information corresponding to the calculation statement, the first data flow information including second data migration information of the tensor, the second data migration information indicating that the tensor is migrated from the first storage position to the second storage position through at least one third migration position in sequence;

[0062] Based on the existence of a data transmission path between the first storage location and the second storage location, the first data flow information is generated, and the first data flow information includes the data migration information associated with the tensor.

[0063] Optionally, in an optional design of the fifth aspect, the third migration location is an on-chip cache.

[0064] Optionally, in an optional design of the fifth aspect, the method further includes:

[0065] The first storage location and the second storage location are determined according to the type of the computing unit.

[0066] In a sixth aspect, the present application provides a compiler, comprising:

[0067] an acquisition unit, configured to acquire a first intermediate expression IR, wherein the first IR includes a calculation statement, the calculation statement includes a tensor and an operator, the operation represented by the operator is performed based on the calculation unit, the tensor is used to migrate to the calculation unit through a first storage location and a second storage location in sequence, and the tensor is data used to perform the operation represented by the operator;

[0068] A processing unit is used to generate a second IR based on the calculation statement, the second IR includes first data migration information and data splitting information, the first data migration information indicates that the migration path of the tensor includes from the first storage location to the second storage location, and the data splitting information indicates the size of the tensor migrated to the second storage location each time.

[0069] Optionally, in an optional design of the sixth aspect, the first storage location indicates a storage location on an on-chip cache, and the second storage location indicates a storage location on the on-chip cache.

[0070] Optionally, in an optional design of the sixth aspect, the first storage location is a memory, and the second storage location indicates a storage location on an on-chip cache.

[0071] Optionally, in an optional design of the sixth aspect, the tensor corresponds to the target variable in the calculation statement, and the second IR also includes: a value range of the target variable, and a mapping of the target variable to the tensor of the first storage location.

[0072] Optionally, in an optional design of the sixth aspect, the data segmentation information includes a value range of the target variable corresponding to each time the tensor is migrated to the second storage location.

[0073] Optionally, in an optional design of the sixth aspect, the tensor includes a target axis variable, the split tensor includes a split axis variable of the target axis variable, and the data split information includes a size relationship between the split axis variable and the target axis variable each time the tensor is migrated to the second storage location.

[0074] Optionally, in an optional design of the sixth aspect, the second IR includes node information corresponding to the second cache, and the node information includes the first data migration information and the data segmentation information.

[0075] Optionally, in an optional design of the sixth aspect, the second IR also includes: read and write information, and the read and write information indicates that the second storage location reads the tensor from the first storage location.

[0076] Optionally, in an optional design of the sixth aspect, the processing unit is further used for:

[0077] Generate first data flow information corresponding to the calculation statement, the first data flow information including first data migration information associated with the tensor, the first data migration information indicating that the tensor is migrated from the first storage location to the second storage location.

[0078] Optionally, in an optional design of the sixth aspect, the processing unit is specifically used for:

[0079] Generate first data flow information corresponding to the calculation statement, the first data flow information including second data migration information of the tensor, the second data migration information indicating that the tensor is migrated from the first storage position to the second storage position through at least one third migration position in sequence;

[0080] Based on the existence of a data transmission path between the first storage location and the second storage location, the first data flow information is generated, and the first data flow information includes the data migration information associated with the tensor.

[0081] Optionally, in an optional design of the sixth aspect, the third migration location is an on-chip cache.

[0082] Optionally, in an optional design of the sixth aspect, the processing unit is further used for:

[0083] The first storage location and the second storage location are determined according to the type of the computing unit.

[0084] In a seventh aspect, the present application provides a computer system, comprising: a processor and a memory;

[0085] The processor is electrically connected to the memory;

[0086] The processor is used to call the code in the memory to execute any method as described in the fifth aspect above.

[0087] In an eighth aspect, the present application provides a computer storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes any method described in the fifth aspect above.

[0088] The embodiment of the present application provides a method for constructing an intermediate expression, including: obtaining a first intermediate expression IR, the first IR including a calculation statement, the calculation statement including a tensor and an operator, wherein the operation represented by the operator is executed based on a calculation unit, the calculation unit is used to perform operations on data migrated through a first storage location and a second storage location in sequence, and the tensor is the data used to execute the operation represented by the operator; generating a second IR based on the calculation statement, the second IR including first data migration information and data segmentation information, the first data migration information indicating that the first migration path of the tensor includes from the first storage location to the second storage location, and the data segmentation information indicates the size of the tensor when the tensor migrates to the second storage location. In the above manner, the compiler can construct an intermediate expression that can express the migration of tensors between different storage locations (on-chip cache or memory) (including the direction of migration and the size of migration), which can be applied to the construction of IR on AI chips for DSA. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 A schematic diagram of the application architecture of an embodiment of the present application;

[0090] Figure 2 A flowchart illustrating a method for constructing an intermediate expression provided in an embodiment of the present application;

[0091] Figure 3 A schematic diagram of the structure of an AI core in an AI chip provided in an embodiment of the present application;

[0092] Figure 4 An abstract illustration of data flow information provided in an embodiment of the present application;

[0093] Figure 5 A schematic diagram of an embodiment of a method for constructing an intermediate expression provided in an embodiment of the present application;

[0094] Figure 6 An abstract illustration of data flow information provided in an embodiment of the present application;

[0095] Figure 7 An abstract illustration of data flow information provided in an embodiment of the present application;

[0096] Figure 8 An abstract illustration of data flow information provided in an embodiment of the present application;

[0097] Fig. 9 A schematic diagram of the structure of a compiler provided in an embodiment of the present application. DETAILED DESCRIPTION

[0098] The embodiments of the present application provide a method, compiler, and server for constructing an intermediate expression, which can construct an intermediate expression that can express the migration of tensors between different storage locations (on-chip cache or memory) (including the direction of migration and the size of migration).

[0099] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0100] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0101] Reference Figure 1 , Figure 1 This is a schematic diagram of the application architecture of an embodiment of the present application, such as Figure 1As shown in, the present application can be applied in a server, wherein the server may include an AI training and reasoning framework of the software part, and the AI ​​training and reasoning framework may include a compiler. In an embodiment of the present application, the compiler can obtain source code from the memory and compile the source code into an intermediate expression and a machine language that can be recognized and executed by the AI ​​chip.

[0102] Reference Figure 2 , Figure 2 A flowchart of a method for constructing an intermediate expression provided in an embodiment of the present application is shown as follows: Figure 2 As shown in, the method for constructing the intermediate expression provided by the present application includes:

[0103] 201. The compiler obtains a first intermediate expression IR, where the first IR includes a calculation statement, and the calculation statement includes a tensor and an operator, where the operation represented by the operator is executed based on a calculation unit, and the calculation unit is used to perform operations on data migrated through a first storage location and a second storage location in sequence, and the tensor is data used to execute the operation represented by the operator.

[0104] Optionally, in an embodiment of the present application, the compiler may obtain a first intermediate representation (IR).

[0105] It should be understood that the first IR can be generated by parsing and compiling computer source code. Alternatively, the first IR can be generated by decompiling an existing computer program. Alternatively, the first IR is obtained from the outside. The source code used to generate the first IR can be written using an application editing interface of a high-level editing language. The high-level programming language can be a domain specific language (DSL).

[0106] The first IR can be stored in a memory (external storage or internal storage of the server). In an embodiment of the present application, the compiler can read the first IR from the memory (external storage or internal storage of the server). Specifically, the first IR can be described by a DSL language. In an embodiment of the present application, the DSL language can be: Halide, Graphlt, Spatial, or other customized domain-specific languages. Among them, Halide is applicable to vector and tensor operations, GraphIt is applicable to the field of graph computing, Spatial is applicable to the field of programmable hardware, and customized domain-specific languages ​​are applicable to corresponding customized fields.

[0107] In an embodiment of the present application, the compiler may sequentially traverse each calculation statement in the acquired first IR, and analyze the calculation units to be used by each calculation statement.

[0108] In one embodiment, the computing unit may include at least: a scalar computing unit, a vector computing unit and a tensor cube computing unit.

[0109] In the embodiment of the present application, different computing units will be used for different operator types. For example, the vector computing unit can support addition (add), subtraction (sub), multiplication (mul), reciprocal (rec), exponential (exp), logarithm (log), and quantization operations, etc. The cube computing unit can support convolution operations.

[0110] In the architecture of some AI chips (for example, AI chips based on domain specific architecture (DSA)), many dedicated on-chip caches are used to shorten the distance of data movement to reduce the overhead caused by data movement. With the evolution of the DSA architecture, the increase in computing units and the improvement of operator complexity, the types of on-chip caches have increased exponentially. For example, a versatile tensor accelerator (VTA) has three types of on-chip caches. In other processors, the types of on-chip caches can be 5 or even more, which is not limited in this application.

[0111] Reference Figure 3 , Figure 3 The structure of an AI core in an AI chip provided in an embodiment of the present application is shown as follows: Figure 3 As shown in the figure, the AI ​​core includes multiple on-chip caches (L1 buffer, L0A buffer, L0B buffer, L0C buffer, Unified buffer), multiple computing units (cube computing unit, vector computing unit), data transfer processing unit and bus interface components. Among them, the bus interface component can obtain the tensor in the AI ​​chip memory, and migrate the tensor to the corresponding computing unit through the above-mentioned on-chip cache to realize the corresponding operation.

[0112] It should be noted that after the computing unit is determined, the migration route of the data (tensor) can be determined based on the specific architecture of the AI ​​core, where the migration route may not be unique, that is, the data can be migrated to the computing unit through multiple migration routes and through different on-chip caches.

[0113] It should be noted that the “migration” in this application may refer to the reading of data. For example, the migration of a tensor from buffer1 to buffer2 may refer to buffer2 reading the tensor in buffer1.

[0114] It should be noted that although Figure 3Not shown, the AI ​​core may also include a data control unit that can control the migration direction of tensors in the on-chip cache.

[0115] It should be noted that Figure 3 The on-chip cache in Figure 1 The memory in the server can be understood as different storage media (the on-chip cache is the storage medium in the AI ​​chip).

[0116] In an embodiment of the present application, the calculation statement of the first IR may include at least one tensor and at least one tensor-related operator, where the tensor is data, which can be understood as a multi-dimensional vector, for example.

[0117] In the embodiment of the present application, an operator can represent a certain operation rule. For example, if a certain operator is a multiplication operation, the operator needs to be executed based on the vector computing unit, and the migration of the tensor from the memory to the vector computing unit needs to pass through the memory, the bus interface component, and the unified buffer in sequence before it can be migrated to the vector computing unit. That is, the vector computing is used to obtain the tensor through the migration of the memory and the unified buffer in sequence, thereby realizing the operation of the operator.

[0118] Taking the first IR as a halide IR as an example, illustratively, the first IR may refer to the following IR schematic:

[0119] #First IR indication

[0120] for(i1, 0, 64){

[0121] for(i2, 0, 64){

[0122] result(i1,i2)=input_1(i1,i2)*input_2(i1,i2)}}

[0123] Among them, input_1(i1,i2) represents tensor A, input_2(i1,i2) represents tensor B, and result(i1,i2) is the result. for(i1,0,64) represents a for loop, and the traversal rule is that the target variable i1 starts from 0 and accumulates 1 each time, accumulating 64 times. result(i1,i2) is a two-dimensional tensor, which is the product of tensor A and tensor B, and the size of each dimension is 64.

[0124] It should be noted that the target variable can be a loop variable of a calculation statement, wherein a part of the loop variable can be the target variable of the tensor included therein.

[0125] In an embodiment of the present application, the compiler can obtain the calculation statement in the above-mentioned first IR, which includes tensor A, tensor B, tensor (result) and operator (multiplication operation). Then, the compiler can determine that the calculation unit corresponding to the operator (multiplication operation) is a vector calculation unit, and the vector calculation is used to obtain the tensor through memory and unified buffer in sequence, thereby realizing the operation of the operator (multiplication operation).

[0126] Optionally, in an embodiment of the present application, the compiler may generate first data flow information corresponding to the calculation statement, the first data flow information including first data migration information associated with the tensor, the first data migration information indicating that the tensor is migrated from a first storage location to a second storage location.

[0127] Optionally, the first storage location indicates a storage location on an on-chip cache, and the second storage location indicates a storage location on an on-chip cache.

[0128] Optionally, the first storage location indicates a storage location on a memory, and the second storage location indicates a storage location on an on-chip cache.

[0129] It should be noted that the data flow information in the embodiments of the present application can represent the migration of tensors in the AI ​​core. For example, it can be a data stack structure, which can represent the migration path of tensors in the AI ​​core. The present application does not limit the specific implementation method of the data flow information.

[0130] Taking the above as an example, refer to Figure 4 , Figure 4 An abstract representation of data flow information provided in an embodiment of the present application is shown in FIG. Figure 4 As shown in , the compiler can generate the first data flow information corresponding to the calculation statement, and the first data flow information can indicate that tensor A is migrated from the memory to the Unified buffer, tensor B is migrated from the memory to the Unified buffer, tensor A and tensor B are migrated to the Vector calculation unit, and the Vector calculation unit can perform a product operation on tensor A and tensor B to obtain a calculation result result, and then the calculation result result is migrated to the memory.

[0131] Among them, the data migration information related to the tensor A in the first data stream information is: "Memory (A) - UB (A)", it should be noted that, Figure 4 The UB in the above is the Unified buffer. The data migration information related to the tensor B is: "memory (B) - UB (B)". The data migration information related to the tensor result is: "UB (result) - memory (result)".

[0132] 202. The compiler generates a second IR based on the calculation statement, wherein the second IR includes first data migration information and data splitting information, wherein the first data migration information indicates that a first migration path of the tensor includes from the first storage location to the second storage location, and the data splitting information indicates the size of the tensor when the tensor is migrated to the second storage location.

[0133] In the embodiment of the present application, the compiler may generate a second IR, wherein the second IR and the first IR may be intermediate representations described in different languages. Optionally, the second IR may be implemented based on polyhedral compilation technology.

[0134] In an embodiment of the present application, the second IR may be implemented by a scheduling tree based on polyhedron technology, which may include a root node and child nodes, wherein the root node is a domain node, which includes the calculation statement of the operator and the range of the variables therein. A specific range node (band node) is marked as child or schedule in the second IR.

[0135] In an embodiment of the present application, the second IR may include data migration information, and the data migration information indicates that the tensor is migrated from the first storage location to the second storage location.

[0136] Refer to the second IR diagram below:

[0137] #Second IR

[0138]

[0139] The first part represents a statement of the first IR and the range of changes of its related target variables (i1 and i2), and Mark is a marking node, wherein "realize_UB" indicates that the type of the on-chip cache is a unified buffer, that is, Mark: "realize_UB" represents the node information corresponding to the second cache (UB). Accordingly, the node information corresponding to the second cache (UB) includes data migration information and data segmentation information corresponding to the second cache (UB).

[0140] The second part represents the data migration information and data segmentation information of the tensor, that is, the migration mapping relationship from the memory to the on-chip cache UB, where i1, i2 represent the target variables related to the statement, arg0, arg1 represent the axis variables of the tensor, and arg0', arg1' represent the segmentation axis variables of the first tensor.

[0141] It should be noted that since the storage size between on-chip caches may be different, the tensors need to be split so that each subsequent tensor migration process is based on the split tensors.

[0142] Specifically, [i1,i2]->L1read[[[i1,i2]->A[arg0,arg1]]->A_local_L1[arg0',arg1']] represents the mapping relationship of the first tensor from the memory to the on-chip cache UB, and -> represents a layer of mapping relationship. Among them, [[i1,i2]->A[arg0,arg1]] represents the mapping of the target variables i1,i2 to the tensors of the first storage location (memory), [i1,i2]->A[arg0,arg1]]->A_local_L1[arg0',arg1'] represents the mapping of the tensor of the first storage location (memory) to the tensor of the second storage location (UB), and A_local_L1[arg0',arg1'] indicates that the tensor needs to be split every time it is migrated to the second storage location.

[0143] Optionally, the tensor includes multiple dimensions, each dimension corresponds to an axis variable (such as arg0 and arg1 above), and the axis variable is used to indicate the size of the tensor of the corresponding dimension. The data segmentation information includes multiple axis variables and segmentation axis variables corresponding to each axis variable, and the segmentation axis variable is used to indicate the size of the tensor of the corresponding dimension when the tensor is migrated.

[0144] Optionally, the second IR also includes at least one target variable and a value range of each target variable, the axis variable is linearly related to the at least one target variable, and the axis variable represents the tensor size of the corresponding dimension through the at least one target variable and the value range of the at least one target variable.

[0145] In the embodiment of the present application, the tensor size of the dimension corresponding to the axis variable can be expressed by the linear combination and value range of the target variable, for example:

[0146] for(io,0,265){

[0147] for(i1,0,512)

[0148] }

[0149] arg0>=64i0 and 0<=arg0<=255and arg0<=63+64i0 and arg1>=512i1and0<=arg1<=511and arg1<=511+512i1. The above is an illustration of a method for representing the tensor size of the dimensions corresponding to the axis variables arg0 and arg1, where the tensor size of the dimension corresponding to arg0 is expressed by i0, and the tensor size of the dimension corresponding to arg1 is expressed by i1.

[0150] It should be noted that the above-mentioned expression of the axis variables is only for illustration and is not limited here.

[0151] Optionally, the second IR may further include: the size relationship between the segmentation axis variable and the axis variable. For example, the second IR may include the size relationship between the segmentation axis variable arg0' and the axis variable arg0, and the size relationship between the segmentation axis variable arg1' and the axis variable arg1 in each migration process.

[0152] It should be noted that an axis variable in the embodiment of the present application corresponds to a dimension of a tensor, and multiple axis variables can represent the size of a tensor. The split axis variable is a part of the axis variable obtained after the axis variable is split (or divided). Among them, splitting can refer to dividing the axis variable into multiple sub-axis variables, for example, dividing a 64*64 axis variable into two 32*64 split axis variables.

[0153] For example, if 2arg1'=arg1 is specified, it means that during the migration of the tensor, the axis variable arg1 is migrated at half the size of the axis variable arg1 each time (the size of the axis variable depends on the value range of the corresponding target variable). In this case, it takes two times to complete the migration of the axis variable arg1.

[0154] In the above manner, the compiler can construct an intermediate expression that can express the migration of tensors between different storage locations (including the direction of migration and the size of migration).

[0155] It should be noted that, although not shown, the second IR may also include operation statements related to the operators in the calculation statements. Since this application only focuses on the migration process of tensors between on-chip caches, this application will not go into details about the compilation of operators.

[0156] Optionally, in one embodiment, the compiler may obtain the most forward data flow information in the first data flow information (eg Figure 4In it is memory (A) to UE (A), according to the currently acquired data flow information, the insertion position of the data flow information in the second IR is determined, as described in the above embodiment. At this time, the insertion position corresponding to the data flow information is "realizeUB", and the split size of the tensor is determined based on the size of the tensor, which can be the split of the axis variable corresponding to the tensor.

[0157] The compiler obtains the direct migration information of the tensor cache on different slices under the current partition space. Exemplarily, the migration information may be as follows;

[0158] [[io,i1]->A[arg0,arg1]]->A_local_l1[arg0',arg1'];

[0159] 2arg0'=arg0,2arg1'=arg1

[0160] Among them, i0, i1 represent the relevant target variables of the statement; A[arg0, arg1] represents tensor A and its axis variable; A_local_L1[arg0', arg1'] represents tensor A_local_L1 on the on-chip cache L1 and the axis variable of the tensor; 2arg0'=arg0, 2arg1'=arg1 represents the relationship between the axis variables of tensor A_local_L1 and tensor A.

[0161] The compiler can perform the Presburger operation identity based on the above-obtained on-chip cache transfer information. The Presburger operation identity mainly implements a self-to-auto mapping relationship: I = {i->i; i∈S}

[0162] Exemplarily, the following mapping to itself can be obtained:

[0163] [[[io,i1]->A[arg0,arg1]]->A_local_l1[arg0',arg1']]->

[0164] [[[io,i1]->A[arg0,arg1]]->A_local_l1[arg0',arg1']];

[0165] The compiler can perform Presburger operation DFDP based on the obtained self-mapping information. DFDP is a domain multiplication mapping transformation: DFDP = {i->k:j:[i->j]->k∈S}

[0166] Exemplarily, the following multiplication mapping result can be obtained:

[0167] [io,i1]->[[[io,i1]->A[arg0,arg1]]->A_local_l1[arg0',arg1']].

[0168] After obtaining the multiplication mapping result, you can set the read and write information of the range domain;

[0169] Exemplarily, the following data migration information and data segmentation information can be obtained:

[0170] [io,i1]->L1read[[[io,i1]->A[arg0,arg1]]->A_local_l1[arg0',arg1']].

[0171] Among them, L1read represents the reading of data.

[0172] The compiler may insert the above data migration information and data segmentation information into the "realize UB" child node to obtain the second IR.

[0173] The embodiment of the present application provides a method for constructing an intermediate expression, including: obtaining a first intermediate expression IR, the first IR including a calculation statement, the calculation statement including a tensor and an operator, wherein the operation represented by the operator is executed based on a calculation unit, the calculation unit is used to perform operations on data migrated through a first storage location and a second storage location in sequence, and the tensor is the data used to execute the operation represented by the operator; generating a second IR based on the calculation statement, the second IR including first data migration information and data segmentation information, the first data migration information indicating that the first migration path of the tensor includes from the first storage location to the second storage location, and the data segmentation information indicates the size of the tensor when the tensor migrates to the second storage location. In the above manner, the compiler can construct an intermediate expression that can express the migration of tensors between different storage locations (on-chip cache or memory) (including the direction of migration and the size of migration), which can be applied to the construction of IR on AI chips for DSA.

[0174] Reference Figure 5 , Figure 5 A schematic diagram of an embodiment of a method for constructing an intermediate expression provided in an embodiment of the present application is shown in FIG. Figure 5 As shown in , the method of constructing the intermediate expression includes:

[0175] 501. The compiler obtains a first intermediate expression IR.

[0176] For the detailed description of step 501, please refer to step 401, which will not be repeated here.

[0177] 502. The compiler generates first data flow information corresponding to the calculation statement based on the calculation statement, where the first data flow information includes second data migration information of the tensor, and the second data migration information indicates that the tensor is migrated from the first storage position to the second storage position through at least one third migration position in sequence.

[0178] 503. The compiler generates second data flow information based on a data transmission path directly connected between the first storage location and the second storage location, wherein the second data flow information includes the first data migration information.

[0179] Optionally, the compiler may generate first data flow information corresponding to the calculation statement, the first data flow information including second data migration information of the tensor, the second data migration information indicating that the tensor is migrated from the first storage location to the second storage location through at least one third migration location in sequence, and based on the existence of a direct data transmission path between the first storage location and the second storage location, wherein (a direct data transmission path may be understood as: there are no other storage locations between the first storage location and the second storage location, and the tensor may be directly transferred from the first storage location to the second storage location without passing through other storage locations), the first data flow information is generated, and the first data flow information includes the first data migration information. Optionally, the third migration location indicates a storage location on an on-chip cache.

[0180] In the embodiment of the present application, in a scenario where a calculation statement includes multiple operators, the compiler may generate second data flow information corresponding to the calculation statement, such as Figure 6 As shown in FIG. 1 , the initialized second data stream information includes many redundant migration processes, such as Figure 7 As shown in FIG. 1 , the dotted line indicates an optional migration path that can be optimized, for example, in the migration route from UB(F(A)), DDR(F(A)) to L1(F(A)) (UB corresponds to the first storage position, DDR corresponds to at least one second migration position, and L1 corresponds to the second storage position), at this time, referring to Figure 3 , since there is a data transmission path between UB buffer and L1 buffer, UB buffer can directly transfer tensor F(A) to L1 buffer. Correspondingly, Figure 8 As shown in , at this time, the first data flow information includes first data migration information associated with the tensor F(A), and the first data migration information indicates that the tensor is migrated from the first storage location (UB buffer) to the second storage location (L1 buffer).

[0181] Optionally, a certain weight may be assigned to each migration, and the higher the weight, the greater the cost of the migration on performance. The data flow weight table may refer to the following table 1:

[0182] Table 1

[0183]

[0184] The weight of DDR represents the quantification of the performance cost of obtaining tensors from memory. The weight of L1 / UB represents the quantification of the performance cost of obtaining tensors from L1 buffer and UB buffer. The weight of L01 / L0B / L0C represents the quantification of the performance cost of obtaining tensors from L01 buffer, L0B buffer, and L0C buffer.

[0185] It should be noted that the compiler also needs to mark the attribute of each edge in the data flow information as data movement or data calculation. Only the edges of data movement can be optimized in this processing flow, while the edges of data calculation cannot be optimized during optimization.

[0186] For example, Figure 6 As shown in FIG, the route from DDR(A) to UB(F(A)) actually involves data calculation on tensor A (obtaining F(A)). Therefore, the migration route from DDR(A) to UB(F(A)) cannot be deleted.

[0187] The compiler can traverse all nodes in the data flow graph and calculate the set of all start nodes with zero in-degree (e.g. Figure 7 DDR(A), DDR(B), and DDR(D) in ) and the end node set with zero out-degree (e.g. Figure 7 DDR(RES) in ), and obtain all feasible path tables of the data flow graph, which can represent possible paths from the start node set to the end node set.

[0188] The weight corresponding to each path is calculated, and the weight is the sum of the weights of all migration processes on the path, and the data flow information corresponding to the path with the smallest sum of weights is determined as the first data flow information. Figure 8 As shown in Figure 8 The optimized first data flow graph is shown.

[0189] In an embodiment of the present application, a first intermediate expression IR is obtained, wherein the first IR includes a calculation statement, and the calculation statement includes a tensor; based on the calculation statement, first data flow information corresponding to the calculation statement is generated, wherein the first data flow information includes second data migration information of the tensor, and the second data migration information indicates that the tensor is migrated from the first storage location to the second storage location through at least one third migration location in sequence; based on the existence of a direct data transmission path between the first storage location and the second storage location, second data flow information is generated, wherein the second data flow information includes the first data migration information. In the above manner, redundant migration paths in the second data migration information are deleted, and on the basis of ensuring that the tensor can be migrated to the second storage location, the cost of moving the tensor is reduced, thereby reducing the system overhead.

[0190] Reference Fig. 9 , Fig. 9 A schematic diagram of the structure of a compiler provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown in , the compiler includes:

[0191] An acquisition unit 901 is used to acquire a first intermediate expression IR, where the first IR includes a calculation statement, the calculation statement includes a tensor and an operator, wherein the operation represented by the operator is performed based on a calculation unit, the calculation unit is used to perform an operation on data migrated through the first storage location and the second storage location in sequence, and the tensor is data used to perform the operation represented by the operator;

[0192] Processing unit 902 is used to generate a second IR based on the calculation statement, the second IR includes first data migration information and data splitting information, the first data migration information indicates that the first migration path of the tensor includes from the first storage location to the second storage location, and the data splitting information indicates the size of the tensor when the tensor is migrated to the second storage location.

[0193] Optionally, the computing unit is further used to perform operations on data passing through a second migration path, wherein the second migration path includes data from the first storage location, through at least one third storage location, to the second storage location, and the first migration path and the second migration path are different data migration paths.

[0194] Optionally, the at least one third storage location indicates a storage location on an on-chip cache.

[0195] Optionally, the processing module is further used to: generate first data flow information corresponding to the calculation statement based on the calculation statement, the first data flow information including second data migration information of the tensor, and the second data migration information indicates that the tensor is migrated from the first storage position to the second storage position through at least one third migration position in sequence;

[0196] Based on the existence of a direct data transmission path between the first storage location and the second storage location, second data flow information is generated, wherein the second data flow information includes the first data migration information.

[0197] Optionally, the first storage location indicates a storage location on an on-chip cache, and the second storage location indicates a storage location on an on-chip cache.

[0198] Optionally, the first storage location indicates a memory, and the second storage location indicates a storage location on an on-chip cache.

[0199] Optionally, the tensor includes multiple dimensions, each dimension corresponds to an axis variable, and the axis variable is used to represent the size of the tensor of the corresponding dimension. The data splitting information includes multiple axis variables and a splitting axis variable corresponding to each axis variable, and the splitting axis variable is used to represent the size of the tensor of the corresponding dimension when the tensor is migrated.

[0200] Optionally, the second IR also includes at least one target variable and a value range of each target variable, the axis variable is linearly related to the at least one target variable, and the axis variable represents the tensor size of the corresponding dimension through the at least one target variable and the value range of the at least one target variable.

[0201] Optionally, the second IR is an IR with a tree structure, the tree structure including a root node and child nodes, the root node corresponds to the calculation statement, and the child node corresponds to the second storage location, wherein the second IR includes information of the child node corresponding to the second storage location, and the information of the child node corresponding to the second storage location includes the first data migration information and the data segmentation information.

[0202] Optionally, the second IR also includes: data movement information, wherein the data movement information indicates that the tensor is moved from the first storage location to the second storage location.

[0203] Optionally, the processing unit is further used for:

[0204] The first storage location and the second storage location are determined according to the type of the computing unit.

[0205] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0206] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0207] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0208] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for constructing an intermediate expression, characterized in that: include: Obtaining a first IR, wherein the first IR includes a calculation statement, wherein the calculation statement includes a tensor and an operator, wherein the operation represented by the operator is performed based on a calculation unit, wherein the calculation unit is used to perform an operation on data migrated through a first storage location and a second storage location in sequence, wherein the tensor is data used to perform the operation represented by the operator; the calculation unit is further used to perform an operation on data passing through a second migration path, wherein the second migration path includes a path from the first storage location, through at least one third storage location, to the second storage location, wherein the first migration path and the second migration path are different data migration paths; A second IR is generated based on the calculation statement, and the second IR includes first data migration information and data splitting information. The first data migration information indicates that the first migration path of the tensor includes from the first storage location to the second storage location, and the data splitting information indicates the size of the tensor when the tensor is migrated to the second storage location.

2. The method according to claim 1, characterized in that The at least one third storage location indicates a storage location on the on-chip cache.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: generating first data flow information corresponding to the calculation statement based on the calculation statement, the first data flow information including second data migration information of the tensor, the second data migration information indicating that the tensor is migrated from the first storage position to the second storage position through at least one third migration position in sequence; Based on the existence of a direct data transmission path between the first storage location and the second storage location, second data flow information is generated, wherein the second data flow information includes the first data migration information.

4. The method according to claim 1 or 2, characterized in that: The first storage location indicates a storage location on an on-chip cache, and the second storage location indicates a storage location on an on-chip cache.

5. The method according to claim 1 or 2, characterized in that: The first storage location indicates a memory, and the second storage location indicates a storage location on an on-chip cache.

6. The method according to claim 1 or 2, characterized in that: The tensor includes multiple dimensions, each dimension corresponds to an axis variable, and the axis variable is used to represent the size of the tensor of the corresponding dimension. The data splitting information includes multiple axis variables and a splitting axis variable corresponding to each axis variable, and the splitting axis variable is used to represent the size of the tensor of the corresponding dimension when the tensor is migrated.

7. The method according to claim 6, characterized in that The second IR also includes at least one target variable and a value range of each target variable. The axis variable is linearly related to the at least one target variable. The axis variable represents the tensor size of the corresponding dimension through the at least one target variable and the value range of the at least one target variable.

8. The method according to claim 1 or 2, characterized in that: The second IR is an IR of a tree structure, the tree structure including a root node and child nodes, the root node corresponds to the calculation statement, and the child node corresponds to the second storage location, wherein the second IR includes information of the child node corresponding to the second storage location, and the information of the child node corresponding to the second storage location includes the first data migration information and the data segmentation information.

9. The method according to claim 1 or 2, characterized in that: The second IR also includes: data movement information, and the data movement information indicates that the tensor is moved from the first storage location to the second storage location.

10. The method according to claim 1 or 2, characterized in that: The method further comprises: The first storage location and the second storage location are determined according to the type of the computing unit.

11. A compiler, characterized in that include: an acquisition unit, configured to acquire a first IR, wherein the first IR includes a calculation statement, wherein the calculation statement includes a tensor and an operator, wherein the operation represented by the operator is performed based on the calculation unit, and the calculation unit is configured to perform operations on data migrated through a first storage location and a second storage location in sequence, wherein the tensor is data used to perform the operation represented by the operator; the calculation unit is further configured to perform operations on data passing through a second migration path, wherein the second migration path includes a path from the first storage location, through at least one third storage location, to the second storage location, wherein the first migration path and the second migration path are different data migration paths; A processing unit is used to generate a second IR based on the calculation statement, the second IR includes first data migration information and data splitting information, the first data migration information indicates that the first migration path of the tensor includes from the first storage location to the second storage location, and the data splitting information indicates the size of the tensor when the tensor is migrated to the second storage location.

12. The compiler according to claim 11, characterized in that The at least one third storage location indicates a storage location on the on-chip cache.

13. The compiler according to claim 11 or 12, characterized in that: The processing unit is further configured to: generate first data flow information corresponding to the calculation statement based on the calculation statement, the first data flow information including second data migration information of the tensor, the second data migration information indicating that the tensor is migrated from the first storage position to the second storage position through at least one third migration position in sequence; Based on the existence of a direct data transmission path between the first storage location and the second storage location, second data flow information is generated, wherein the second data flow information includes the first data migration information.

14. The compiler according to claim 11 or 12, characterized in that: The first storage location indicates a storage location on an on-chip cache, and the second storage location indicates a storage location on an on-chip cache.

15. The compiler according to claim 11 or 12, characterized in that: The first storage location indicates a memory, and the second storage location indicates a storage location on an on-chip cache.

16. The compiler according to claim 11 or 12, characterized in that: The tensor includes multiple dimensions, each dimension corresponds to an axis variable, and the axis variable is used to represent the size of the tensor of the corresponding dimension. The data splitting information includes multiple axis variables and a splitting axis variable corresponding to each axis variable, and the splitting axis variable is used to represent the size of the tensor of the corresponding dimension when the tensor is migrated.

17. The compiler according to claim 16, characterized in that The second IR also includes at least one target variable and a value range of each target variable. The axis variable is linearly related to the at least one target variable. The axis variable represents the tensor size of the corresponding dimension through the at least one target variable and the value range of the at least one target variable.

18. The compiler according to claim 11 or 12, characterized in that: The second IR is an IR of a tree structure, the tree structure including a root node and child nodes, the root node corresponds to the calculation statement, and the child node corresponds to the second storage location, wherein the second IR includes information of the child node corresponding to the second storage location, and the information of the child node corresponding to the second storage location includes the first data migration information and the data segmentation information.

19. The compiler according to claim 11 or 12, characterized in that: The second IR also includes: data movement information, and the data movement information indicates that the tensor is moved from the first storage location to the second storage location.

20. The compiler according to claim 11 or 12, characterized in that: The processing unit is also used for: The first storage location and the second storage location are determined according to the type of the computing unit.

21. A computer system, characterized in that: include: Processor and memory; The processor is electrically connected to the memory; The processor is used to call the code in the memory to execute the method according to any one of claims 1 to 10.

22. A computer storage medium, characterized in that The computer storage medium stores instructions, which, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 10.

Images