Method and device for compiling operator

Abstract

The invention relates to the field of artificial intelligence, and provides an operator compiling method and device, and the method comprises the steps: obtaining a to-be-compiled operator; determining at least one first scalar used in the first instruction and at least one second scalar used in the second instruction; determining at least one third instruction and at least one fourth instruction in the operator to be compiled; a first target operator is generated by the first instruction and the at least one third instruction, and a second target operator is generated by the second instruction and the at least one fourth instruction. According to the method, by determining the scalar used in each instruction executed by the specific calculation module and the instructions which have actual influence on assignment of the scalar before the scalar is used by the instructions, redundant instructions which have no influence on a calculation result can be eliminated from operators compiled by the specific calculation module; therefore, the calculation efficiency of the compiled operator is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a method and device for compiling operators. Background Art

[0002] With the development of artificial intelligence (AI) technology, the types and amount of calculations that AI chips need to complete are increasing. Therefore, the current AI chips generally have a variety of hardware units dedicated to specific computing tasks, such as matrix multiplication calculation modules for matrix calculations and vector calculation modules for vector calculations. The current mainstream AI chips use a tightly coupled architecture. There is a direct data path between the matrix multiplication calculation module and the vector calculation module, but they cannot be scheduled at the same time. Therefore, there is a trend towards a loosely coupled architecture in the future.

[0003] The hardware structure of AI chips is adjusted from a tightly coupled architecture to a loosely coupled architecture, and the instructions in the operators need to be adjusted accordingly according to the changes in the hardware structure. At present, most operators are developed for AI chips with tightly coupled architectures. Compiling the operators so that the instructions in the operators can be adjusted to be executed in AI chips with loosely coupled architectures is an effective way to make operators developed for tightly coupled architectures compatible with loosely coupled architectures.

[0004] However, the existing operator compilation method only removes specific instructions from the operator that cannot be executed in the current computing module, and uses the removed operators as the compiled operators of the current computing module. The above method causes instructions that do not need to be executed by a specific computing module to appear repeatedly in the compiled operators of different computing modules, resulting in redundant calculations and reducing the computational efficiency of the compiled operators. Therefore, how to improve the computational efficiency of the compiled operators has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a method and device for compiling operators. By establishing a dependency relationship between instructions and scalars executed by a specific computing module, it is possible to determine the scalars used in each instruction executed by the specific computing module, as well as the instructions that have an actual impact on the assignment of these scalars before being used by these instructions. This allows the operator compiled by the specific computing module to eliminate redundant instructions that have no impact on the calculation results, thereby improving the calculation efficiency of the compiled operator.

[0006] In a first aspect, a method for compiling an operator is provided, comprising: obtaining an operator to be compiled, the operator to be compiled comprising a first instruction executed by a first computing module in an artificial intelligence chip and a second instruction executed by a second computing module in the artificial intelligence chip, the first computing module and the second computing module performing different types of computing tasks; determining at least one first scalar used in the first instruction and at least one second scalar used in the second instruction; determining at least one third instruction and at least one fourth instruction in the operator to be compiled, the at least one third instruction being an instruction that last assigns a value to each first scalar in at least one first scalar before the execution sequence of the first instruction, and the at least one fourth instruction being an instruction that last assigns a value to each second scalar in at least one second scalar before the execution sequence of the second instruction; generating a first target operator from the first instruction and the at least one third instruction, generating a second target operator from the second instruction and the at least one fourth instruction, the instructions in the first target operator being executed by the first computing module, and the instructions in the second target operator being executed by the second computing module.

[0007] Optionally, the operator to be compiled may be an operator of an artificial intelligence (AI) chip suitable for a tightly coupled architecture, and the target operator may be an operator of an AI chip suitable for a loosely coupled architecture.

[0008] Optionally, the AI ​​chip may include, but is not limited to, a graphics processing unit (GPU), a field programmable gate array (FPGA), and an application specific integrated circuit (ASIC).

[0009] Optionally, the first calculation module may be a matrix multiplication calculation module, and the second calculation module may be a vector calculation module, or the first calculation module may be a vector calculation module, and the second calculation module may be a matrix multiplication calculation module.

[0010] According to the technical solution provided in the present application, by establishing a dependency relationship between instructions and scalars executed by a specific computing module, it is possible to determine the scalars used in each instruction executed by the specific computing module, as well as the instructions that have an actual impact on the assignment of these scalars before being used by these instructions, so that redundant instructions that have no impact on the calculation results can be eliminated from the operator compiled by the specific computing module, thereby improving the computational efficiency of the compiled operator.

[0011] In combination with the first aspect, in certain implementations of the first aspect, the method also includes: determining a fifth instruction from the operator to be compiled, the fifth instruction being used to instruct the interaction data to be transmitted between the first cache area of ​​the first computing module and the second cache area of ​​the second computing module; determining a sixth instruction and a seventh instruction based on the fifth instruction, the sixth instruction being used to instruct the interaction data to be written into the global memory of the artificial intelligence chip, and the seventh instruction being used to instruct the interaction data to be read from the global memory; adding the sixth instruction to the first target operator and adding the seventh instruction to the second target operator, and / or adding the sixth instruction to the second target operator and adding the seventh instruction to the first target operator.

[0012] According to the above technical solution, data that was originally exchanged directly between two different types of computing modules through their respective cache areas under a tightly coupled architecture is transferred through the global memory in the chip, which can solve the problem of non-existence of some data paths caused by the adjustment of the hardware architecture from tight coupling to loose coupling, thereby improving the applicability of the compiled operator.

[0013] In combination with the first aspect, before determining the sixth instruction and the seventh instruction according to the fifth instruction, the method also includes: establishing a mirror area in the working space of the global memory, the mirror area is a mapping of the cache area with the smallest capacity between the first cache area and the second cache area; determining the sixth instruction and the seventh instruction according to the fifth instruction, including: determining the sixth instruction and the seventh instruction according to the fifth instruction and the mirror area, the sixth instruction is used to instruct to write interactive data into the mirror area, and the seventh instruction is used to instruct to read interactive data from the mirror area.

[0014] According to the above technical solution, a mirror area of ​​the cache area in the computing module is established in the working space of the global memory, and the capacity of the mirror area is determined according to the capacity of the cache area, so that only a fixed size of capacity in the global memory is needed to meet the needs of data transfer between two different computing modules, avoiding the need to re-apply for space in the global memory for each data transfer, thereby saving the memory space of the AI ​​chip occupied by the compiled operator during calculation.

[0015] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: determining at least one third scalar according to the first instruction and the second instruction, the at least one third scalar being a scalar included in both the at least one first scalar and the at least one second scalar; establishing a mirror area in the working space of the global memory, including: establishing the mirror area according to the size of the first cache area, the size of the second cache area, and the size of each third scalar in the at least one third scalar, wherein the size of the mirror area is the sum of the size of the cache area with the smallest capacity between the first cache area and the second cache area and the size of each third scalar in the at least one third scalar.

[0016] According to the above technical solution, by establishing a mirror area in the working space of the global memory for the transfer of associated scalars in instructions executed by different computing modules, synchronous computing between heterogeneous cores can be achieved, thereby improving the applicability of compiled operators.

[0017] In combination with the first aspect, in certain implementations of the first aspect, determining at least one third instruction and at least one fourth instruction in the operator to be compiled includes: when the current instruction is not a loop statement, querying the instructions in the operator to be compiled one by one from the current instruction position upward, wherein the current instruction is the first instruction or the second instruction; determining at least one eighth instruction, wherein the at least one eighth instruction is an instruction that assigns a value to the current scalar and appears for the first time in the query path under each first condition in at least one first condition, and the at least one first condition is a condition for the execution of the current instruction, wherein the current scalar is at least one first scalar used in the first instruction or at least one second scalar used in the second instruction; determining at least one eighth instruction as a target instruction, wherein the target instruction is at least one third instruction corresponding to the first instruction or at least one fourth instruction corresponding to the second instruction.

[0018] According to the above technical solution, when the current instruction is outside the loop statement, by traversing upward from the current statement according to the arrangement order of the instruction statements in the operator, it is possible to find the instruction for the last assignment of the current scalar that does not cross the conditional statement with the current instruction, and only retain these found instructions in the scalar assignment instructions of the compiled operator, discarding redundant calculations that do not affect the calculation results of the current instruction, thereby improving the calculation efficiency of the compiled operator.

[0019] In combination with the first aspect, in certain implementations of the first aspect, determining at least one third instruction and at least one fourth instruction in the operator to be compiled includes: when the current instruction is a loop statement, querying the instructions in the operator to be compiled one by one from the current instruction position upward, wherein the current instruction is the first instruction or the second instruction; determining at least one eighth instruction, wherein the at least one eighth instruction is an instruction that assigns a value to the current scalar that appears for the first time in the query process under each first condition in at least one first condition, wherein the at least one first condition is a condition for the execution of the current instruction, wherein the current scalar is at least one first scalar used in the first instruction or at least one second scalar used in the second instruction; and determining at least one eighth instruction. In the case where each eighth instruction in the instructions belongs to a loop statement at the same level as the current instruction, at least one eighth instruction is determined as a target instruction, wherein the target instruction is at least one third instruction corresponding to the first instruction or at least one fourth instruction corresponding to the second instruction; in the case where at least one eighth instruction does not belong to a loop statement at the same level as the current instruction, loop statements belonging to the same level as the current instruction are searched one by one from the current instruction position downward; at least one ninth instruction is determined, and at least one eighth instruction is the instruction that assigns a value to the current scalar and appears for the last time in the query process under each first condition in at least one first condition; at least one eighth instruction and at least one ninth instruction are determined as target instructions.

[0020] According to the above technical solution, when the current instruction is located within a loop statement, in addition to traversing upward from the current statement according to the arrangement order of the instruction statements in the operator, it also traverses downward within the loop statement to which the current statement belongs. Through the characteristics of the cyclic execution of the loop statement, it is possible to find the instruction that is actually the last assignment of the current scalar in the execution order, and only retain these found instructions in the scalar assignment instructions of the compiled operator, discarding redundant calculations that do not affect the calculation results of the current instruction, thereby improving the calculation efficiency of the compiled operator.

[0021] In a second aspect, a device for compiling an operator is provided, comprising: an acquisition module, used to acquire an operator to be compiled, wherein the operator to be compiled includes a first instruction executed by a first computing module in an artificial intelligence chip and a second instruction executed by a second computing module in the artificial intelligence chip, and the first computing module and the second computing module perform different types of computing tasks; a determination module, used to determine at least one first scalar used in the first instruction and at least one second scalar used in the second instruction; an identification module, used to determine at least one third instruction and at least one fourth instruction in the operator to be compiled, wherein the at least one third instruction is an instruction that last assigns a value to each first scalar in at least one first scalar before the execution sequence of the first instruction, and the at least one fourth instruction is an instruction that last assigns a value to each second scalar in at least one second scalar before the execution sequence of the second instruction; a generation module, used to generate a first target operator from the first instruction and the at least one third instruction, and generate a second target operator from the second instruction and the at least one fourth instruction, wherein the instructions in the first target operator are executed by the first computing module, and the instructions in the second target operator are executed by the second computing module.

[0022] In combination with the second aspect, in certain implementations of the second aspect, the device also includes a transfer module, which is used to: determine a fifth instruction from the operator to be compiled, the fifth instruction is used to instruct the interaction data to be transmitted between the first cache area of ​​the first computing module and the second cache area of ​​the second computing module; determine a sixth instruction and a seventh instruction according to the fifth instruction, the sixth instruction is used to instruct the interaction data to be written to the global memory of the artificial intelligence chip, and the seventh instruction is used to instruct the interaction data to be read from the global memory; add the sixth instruction to the first target operator and add the seventh instruction to the second target operator, and / or add the sixth instruction to the second target operator and add the seventh instruction to the first target operator.

[0023] In combination with the second aspect, in certain implementations of the second aspect, the device also includes a mapping module, which is used to: before determining the sixth instruction and the seventh instruction according to the fifth instruction, establish a mirror area in the working space of the global memory, and the mirror area is a mapping of the cache area with the smallest capacity between the first cache area and the second cache area; a transfer module, which is used to: determine the sixth instruction and the seventh instruction according to the fifth instruction and the mirror area, the sixth instruction is used to instruct to write the interactive data into the mirror area, and the seventh instruction is used to instruct to read the interactive data from the mirror area.

[0024] In combination with the second aspect, in certain implementations of the second aspect, the transport module is further used to: determine at least one third scalar according to the first instruction and the second instruction, wherein the at least one third scalar is a scalar included in both the at least one first scalar and the at least one second scalar; and the mapping module is used to: establish a mirror area according to the size of the first cache area, the size of the second cache area, and the size of each third scalar in the at least one third scalar, wherein the size of the mirror area is the sum of the size of the cache area with the smallest capacity in the first cache area and the second cache area and the size of each third scalar in the at least one third scalar.

[0025] In combination with the second aspect, in certain implementations of the second aspect, the identification module is used to: when the current instruction is not a loop statement, query the instructions in the operator to be compiled one by one from the current instruction position upward, wherein the current instruction is the first instruction or the second instruction; determine at least one eighth instruction, at least one eighth instruction is the instruction that assigns a value to the current scalar that appears for the first time in the query path under each first condition in at least one first condition, and at least one first condition is a condition for the execution of the current instruction, wherein the current scalar is at least one first scalar used in the first instruction or at least one second scalar used in the second instruction; determine at least one eighth instruction as a target instruction, wherein the target instruction is at least one third instruction corresponding to the first instruction or at least one fourth instruction corresponding to the second instruction.

[0026] In combination with the second aspect, in certain implementations of the second aspect, the identification module is used to: when the current instruction is a loop statement, query the instructions in the operator to be compiled one by one from the current instruction position upward, wherein the current instruction is the first instruction or the second instruction; determine at least one eighth instruction, at least one eighth instruction is the instruction that assigns a value to the current scalar that appears for the first time during the query process under each first condition in at least one first condition, and at least one first condition is a condition for the execution of the current instruction, wherein the current scalar is at least one first scalar used in the first instruction or at least one second scalar used in the second instruction; when each eighth instruction in at least one eighth instruction belongs to a loop statement at the same level as the current instruction, determine at least one eighth instruction as a target instruction, wherein the target instruction is at least one third instruction corresponding to the first instruction or at least one fourth instruction corresponding to the second instruction; when at least one eighth instruction does not belong to a loop statement at the same level as the current instruction, query the loop statements at the same level as the current instruction from the current instruction position downward one by one; determine at least one ninth instruction, at least one eighth instruction is the instruction that assigns a value to the current scalar that appears for the last time during the query process under each first condition in at least one first condition; determine at least one eighth instruction and at least one ninth instruction as target instructions.

[0027] In combination with the second aspect, in certain implementations of the second aspect, the artificial intelligence chip is a graphics processor, a field programmable gate array, or an application-specific integrated circuit.

[0028] In combination with the second aspect, in some implementations of the second aspect, the first computing module is a matrix multiplication computing module, and / or the second computing module is a vector computing module.

[0029] In a third aspect, a computing device is provided, comprising a processor and a memory, wherein the memory is used to store instructions, and the processor is used to call and execute the instructions from the memory, so that the computing device executes the method in the first aspect or any possible implementation of the first aspect.

[0030] Optionally, the processor may be a general-purpose processor, which may be implemented by hardware or software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor may be a general-purpose processor, which is implemented by reading software codes stored in a memory, which may be integrated in the processor or may be located outside the processor and exist independently.

[0031] In a fourth aspect, a chip is provided, which obtains instructions and executes the instructions to implement the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0032] Optionally, as an implementation, the chip includes a processor and a data interface, and the processor reads instructions stored in the memory through the data interface to execute the method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0033] Optionally, as an implementation method, the chip may also include a memory, in which instructions are stored, and the processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to execute the method in the above-mentioned first aspect or any possible implementation method of the first aspect.

[0034] In a fifth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device, the computing device executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0035] In a sixth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect.

[0036] By way of example, these computer-readable storage media include, but are not limited to, one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), Flash memory, electrically EPROM (EEPROM), and hard drive.

[0037] Optionally, as an implementation manner, the above-mentioned storage medium may specifically be a non-volatile storage medium. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a structural diagram of a tightly coupled architecture AI chip.

[0039] Figure 2 It is a structural diagram of a loosely coupled architecture AI chip.

[0040] Figure 3 The present invention is a schematic flowchart of an operator compilation method that is compatible with a loosely coupled architecture from a tightly coupled architecture in a traditional solution.

[0041] Figure 4 The present invention is a schematic flowchart of an operator compilation method compatible with a tightly coupled architecture to a loosely coupled architecture provided in an embodiment of the present application.

[0042] Figure 5 It is a schematic flowchart of a method for compiling an operator provided in an embodiment of the present application.

[0043] Figure 6 It is a schematic diagram of the change of the data transfer process when the tightly coupled architecture is compatible with the loosely coupled architecture in an embodiment of the present application.

[0044] Figure 7 It is a schematic structural block diagram of a compilation operator device provided in an embodiment of the present application.

[0045] Figure 8 It is a schematic structural block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0047] The present application will present various aspects, embodiments or features around a system including multiple devices, components, modules, etc. It should be understood and appreciated that each system may include additional devices, components, modules, etc., and / or may not include all devices, components, modules, etc. discussed in conjunction with the figures. In addition, combinations of these schemes may also be used.

[0048] In addition, in the embodiments of the present application, words such as "exemplary" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present concepts in a concrete way.

[0049] In the embodiments of the present application, "corresponding (corresponding, relevant)" and "corresponding (corresponding)" can sometimes be used interchangeably. It should be pointed out that when the distinction between them is not emphasized, the meanings they intend to express are consistent.

[0050] The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

[0051] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0052] In the present application, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: including the existence of A alone, the existence of A and B at the same time, and the existence of B alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0053] To facilitate understanding, relevant terms and concepts that may be involved in the embodiments of the present application are first introduced below.

[0054] 1. Artificial intelligence (AI) chip: AI chip is also called AI accelerator or computing card, which is a module specially used to handle a large number of computing tasks in artificial intelligence applications (other non-computing tasks are still handled by CPU). At present, AI chips are mainly divided into graphics processing unit (GPU), field programmable gate array (FPGA) and application specific integrated circuit (ASIC). The types of computing tasks that AI chips can handle include but are not limited to scalar calculations, vector calculations and matrix calculations, among which the terms such as scalar, vector and matrix represent different data types in the computer field. Different types of computing tasks can be implemented by different hardware units in AI chips. The hardware unit used for specific computing tasks is called a computing module, such as a matrix multiplication computing module for matrix calculations and a vector computing module for vector calculations.

[0055] 2. Scalar: In the computer field, scalar computing refers to a method of computing a single data at a time. Correspondingly, scalar refers to the data type of a single data, such as integers, floating-point numbers, strings, etc.

[0056] 3. Tight coupling: Coupling refers to the degree of mutual connection between two modules. In AI chips, the tightly coupled architecture refers to the direct connection between multiple computing modules responsible for different computing tasks. Figure 1 A schematic diagram of the AI ​​chip structure with a tightly coupled architecture is shown. Figure 1As shown, taking the matrix multiplication calculation module (cube core) and the vector calculation module (vector core) as two hardware units for different computing tasks, there is a directly connected data path between the matrix multiplication calculation module and the vector calculation module under the tightly coupled architecture, so that the matrix multiplication calculation module and the vector calculation module can directly exchange data through their respective buffers. In the tightly coupled mode, the matrix multiplication calculation module and the vector calculation module can be regarded as a subject that cooperates with each other to perform tasks. Therefore, the operator suitable for the tightly coupled architecture can include both matrix calculation instructions executed by the matrix multiplication calculation module and vector calculation instructions executed by the vector calculation module.

[0057] 4. Loose coupling: Loosely coupled architecture means that multiple computing modules responsible for different computing tasks are not directly connected. Figure 2 FIG. 1 shows a schematic diagram of an AI chip structure with a loosely coupled architecture. Figure 2 As shown in the figure, in the loosely coupled architecture, there is no direct data path between the matrix multiplication calculation module and the vector calculation module, but all rely on the global memory in the core memory for data interaction. Therefore, the operator suitable for the tightly coupled architecture is the operator corresponding to a single calculation module, which only includes one of the matrix calculation instructions or vector calculation instructions.

[0058] 5. Operator: An operator is a mapping from a function space to another function space. In the computer field, the computing task on data can be called an operator, which can be implemented by a function. Each function can be called an operator.

[0059] 6. Internal representation (IR): The specific representation of an operator in a computer programming language. The specific calculation process of the function corresponding to the operator is represented by computer code, where each line of code corresponds to a computer instruction. The IR in code form can also be called the syntax tree of the operator. Therefore, the IR and syntax tree that appear below are terms with the same meaning, both representing the specific representation of the operator.

[0060] 7. Operator compilation: Compilation refers to the process of using a compiler to generate a target program from a source program written in a source language. Therefore, operator compilation refers to the process of adjusting the instructions in the operator through the compiler to generate a target operator from the operator to be compiled.

[0061] With the development of AI technology, the types and amount of calculations that AI chips need to complete are increasing. Therefore, the current AI chips generally have a variety of hardware units dedicated to specific computing tasks, such as matrix multiplication calculation modules for matrix calculations and vector calculation modules for vector calculations. The current mainstream AI chips use a tightly coupled architecture, but different types of computing modules directly connected in a tightly coupled architecture cannot be scheduled at the same time, so there is a trend towards a loosely coupled architecture in the future.

[0062] The hardware structure of AI chips is adjusted from a tightly coupled architecture to a loosely coupled architecture, and the instructions in the operators need to be adjusted accordingly according to the changes in the hardware structure. At present, most operators are developed for AI chips with tightly coupled architectures. Compiling the operators so that the instructions in the operators can be adjusted to be executed in AI chips with loosely coupled architectures is an effective way to make operators developed for tightly coupled architectures compatible with loosely coupled architectures.

[0063] Figure 3 FIG. 1 shows a schematic flow chart of a method for compiling operators compatible with a loosely coupled architecture from a tightly coupled architecture in a traditional solution. Figure 3 As shown, the compiler first identifies the specific computing instructions in the operator IR to be compiled (also referred to as the original operator IR) that need to be executed by a specific computing module. For example, the compiler can tag the specific computing instructions to mark which computing module executes the specific instructions, thereby eliminating the specific instructions that cannot be executed by the current computing module during the compilation process, and directly use the eliminated IR as the target operator IR compiled by the current computing module. Optionally, for specific instructions in the operator IR to be compiled that are executed by different computing modules but have an associated relationship, the compiler can also solve the problem that the original direct data path between computing modules in the tightly coupled architecture does not exist in the loosely coupled architecture through data path mapping processing, and solve the scheduling problem of the execution order between specific instructions that need to be executed collaboratively through inter-core synchronization insertion.

[0064] For example, the following code snippet shows the IR of an operator developed for a tightly coupled architecture, that is, an example of the operator IR to be compiled:

[0065]

[0066] Among them, the mix_kernel() function represents the operator implemented by the matrix multiplication calculation module (cube core) and the vector calculation module (vector core). The operator involves two input variables, input0 and input1, and two output variables, output0 and output1. Among them, an integer variable reg is defined, which is a scalar. The Cube() instruction represents the matrix multiplication calculation. For example, Cube(c_buf0,input0,input1,reg) can represent the matrix multiplication calculation based on the variables input0, input1 and reg and the calculation result is stored in the buffer c_buf0 of the cube core. Similarly, Cube(c_buf1,input0,input1,reg) can represent the matrix multiplication calculation based on the variables input0, input1 and reg and the calculation result is stored in the buffer c_buf1 of the cube core. This type of instruction needs to be executed by the cube core. The vector() instruction indicates vector calculation. For example, Vector(output0,v_buf0,reg) indicates that vector calculation is performed based on the data in the variable reg and the buffer v_buf0 of the vector core, and the calculation result is assigned to the output variable output0. Similarly, Vector(output1,v_buf1,reg) indicates that vector calculation is performed based on the data in the variable reg and the buffer v_buf1 of the vector core, and the calculation result is assigned to the output variable output1. Such instructions need to be executed by the vector core. Copy(v_buf0,c_buf0) and Copy(v_buf1,c_buf1) are instructions for exchanging data in the cache between the cube core and the vector core through a directly connected data path in a tightly coupled architecture.

[0067] Therefore, the compiler can tag the Cube() instruction in the operator IR to be compiled with a label that needs to be executed by the cube core, and tag the Vector() instruction with a label that needs to be executed by the vector core. In this way, all Vector() instructions are deleted from the target operator IR executed by the cube core in the generated loosely coupled architecture, and all Cube() instructions are deleted from the target operator IR executed by the vector core in the generated loosely coupled architecture. For example, corresponding to the IR example of the operator developed for the tightly coupled architecture, the compiled IR of the operator that is compatible with the loosely coupled architecture, that is, the target operator IR, is as follows:

[0068] The target operator IR executed by cube core:

[0069]

[0070]

[0071] It can be seen that the target operator IR executed by the cube core does not include the Vector() instruction. In addition, in order to solve the problem that there is no direct data path between the cube core and the vector core in the loosely coupled architecture, the target operator IR will apply for storage space (workspace) workspace0 and workspace1 from the global memory (global memory, gm) in the kernel memory when it is executed. The Copy(workspace0, c_buf0) instruction is used to move the data originally required to be transferred in the cube core's buffer c_buf0 to workspace0. Similarly, the Copy(workspace1, c_buf1) instruction is used to move the data originally required to be transferred in the cube core's buffer c_buf1 to workspace1, that is, Figure 3 The data path mapping process shown in . Furthermore, since the data used by the Cube() instruction and the Vector() instruction in the original operator IR are related to each other, there is a requirement for the execution order of the Cube() instruction and the Vector() instruction. The sync_set(cube,vector) instruction is used to instruct the Cube() instruction in the target operator IR to schedule the execution of the Vector() instruction in other target operator IRs after the execution in each cycle, that is, Figure 3 The inter-nuclear synchronization shown in .

[0072] The corresponding target operator IR executed by the vector core is:

[0073]

[0074] It can be seen that the target operator IR executed by the vector core does not include the Core() instruction. The Copy(v_buf0,workspace0) instruction is used to move the data in workspace0 to the vector core's buffer v_buf0. Similarly, the Copy(v_buf0,workspace0) instruction is used to move the data in workspace1 to the vector core's buffer v_buf1. Figure 3The data path mapping process shown in . The sync_wait(cube,vector) instruction corresponds to the sync_set(cube,vector) instruction in the target operator IR of cubecore, which is used to indicate that the Vector() instruction in the target operator IR needs to wait for the Core() instruction in other target operator IR to execute before executing in each cycle, that is, Figure 3 The inter-nuclear synchronization shown in .

[0075] The above compilation method can make the operator developed for the tightly coupled architecture compatible with the loosely coupled architecture, but the computational efficiency of the target operator obtained after compilation is low. To explain in detail with the above example, in the original operator IR, the integer variable reg has three assignments, namely the instructions reg=0, reg=1 and reg=2. Since the assignment instruction of the integer variable belongs to the scalar operation, it does not need to be executed by a specific computing module and will not be filtered by the above method. Therefore, all the target operator IRs obtained by the above method include the three instructions reg=0, reg=1 and reg=2. However, in the target operator IR executed by the vector core, no matter what kind of calculation has been performed on reg, the instruction reg=2 will be executed before each execution of the Vector() instruction that needs to use reg. In the above example, the value of the scalar reg used when executing the Vector(output0,v_buf0,reg) instruction and the Vector(output1,v_buf1,reg) instruction is always equal to 2, that is, other instructions that calculate the scalar reg have no effect on the calculation results of the Vector(output0,v_buf0,reg) instruction and the Vector(output1,v_buf1,reg) instruction. Therefore, in the above example, when the target operator IR executed by the vector core only includes the two vector calculation instructions Vector(output0,v_buf0,reg) and Vector(output1,v_buf1,reg), the instruction reg=1 is a redundant calculation in the target operator, which reduces the calculation efficiency of the compiled operator.

[0076] In addition, the target operator obtained after the above method is compiled still has the problem of occupying too much memory in the kernel. In combination with the above example, the Copy(workspace0,c_buf0) instruction and the Copy(workspace0,c_buf0) instruction are in the loop statement, so they need to be executed repeatedly. Each time the above instructions are executed, a workspace needs to be requested from the global memory. In fact, each requested workspace is only used once but not released in time, resulting in excessive memory usage in the kernel.

[0077] In view of this, an embodiment of the present application provides a method for compiling operators. By establishing a dependency relationship between instructions and scalars executed by a specific computing module, it is possible to determine the scalars used in each instruction executed by the specific computing module, as well as the instructions that have an actual impact on the assignment of these scalars before being used by these instructions. This allows the operator compiled by the specific computing module to eliminate redundant instructions that have no impact on the calculation results, thereby improving the computational efficiency of the compiled operator.

[0078] Figure 4 FIG. 1 is a schematic flow chart of a method for compiling operators compatible with a tightly coupled architecture to a loosely coupled architecture provided by an embodiment of the present application. Figure 4 As shown, relative to Figure 3 The traditional method shown, the operator compilation method provided in the embodiment of the present application is based on this, and the compiler can also identify the scalar calculation instructions in the operator to be compiled. Specifically, the compiler can traverse each instruction that uses a scalar in the operator to be compiled, and analyze the dependency between the vector calculation instruction and / or the matrix calculation instruction and the scalar calculation instruction according to which scalars are specifically used in each instruction, and label the scalar calculation instruction accordingly according to the analysis result. The compiler assigns a label indicating that it belongs to the vector calculation module to the scalar calculation instruction related to the calculation result of the vector calculation instruction, and assigns a label indicating that it belongs to the matrix multiplication calculation module to the scalar calculation instruction related to the calculation result of the matrix calculation instruction. Further, for scalar calculation instructions that are related to both the calculation result of the vector calculation instruction and the calculation result of the matrix calculation instruction, the compiler will also assign a label that requires heterogeneous core synchronization, and add instructions to store the value of the corresponding scalar to the global memory and / or load the value of the corresponding scalar from the global memory. According to the above method, the compiler can also identify the control logic instructions in the operator to be compiled, which will not be repeated here.

[0079] Optionally, in Figure 4 In the method shown, the compiler can also map the workspace of the global memory to the buffer in the core of the computing module. Specifically, the compiler can find the smaller cache area used in the data exchange process between two different computing modules under the tightly coupled architecture, map it to the workspace of the global memory, and then insert the data transfer instruction to perform the corresponding data transfer.

[0080] Combine the following Figures 5 and 6 , describe in detail the method of compiling operators of this application.

[0081] Figure 5 FIG. 1 is a schematic flow chart of a method for compiling an operator provided in an embodiment of the present application. Figure 5 As shown, the method includes the following steps.

[0082] S510: Obtain an operator to be compiled.

[0083] For example, in step S510, the compiler can obtain the IR of the operator to be compiled. Specifically, in the operator compilation stage, the compiler detects that the current AI chip is a loosely coupled architecture, and the first instruction and the second instruction in the obtained operator IR are two instructions that need to be executed by different types of computing modules, that is, the operator is an operator developed for the AI ​​chip with a tightly coupled architecture, then the operator is determined as the operator to be compiled, and the compilation process to provide compatibility for the operator is entered.

[0084] The first instruction is an instruction executed by a first computing module included in the AI ​​chip, and the second instruction is an instruction executed by a second computing module included in the AI ​​chip, wherein the first computing module and the second computing module perform different types of computing tasks. As an example and not limitation, the first instruction may be a matrix computing instruction, and the corresponding first computing module may be a matrix multiplication computing module; the second instruction may be a vector computing instruction, and the corresponding second computing module may be a vector computing module. The matrix multiplication computing instruction is an instruction for processing data represented in matrix form, and the data represented in matrix form includes but is not limited to digital image data, sample data of multiple feature variables, social network relationship data, etc.; the vector computing instruction is an instruction for processing data represented in vector form, and the data represented in vector form includes but is not limited to natural language data, sample data of a single feature variable, geographic information data, etc.

[0085] It should be understood that in the embodiment of the present application, only the case where the first instruction is a matrix calculation instruction and the second instruction is a vector calculation instruction is used as an example for explanation, but the scope of application of the method of compiling operators provided in the present application is not limited to this. For example, the first instruction may also be a vector calculation instruction, and the corresponding first calculation module is a vector calculation module, and the second instruction may also be a matrix calculation instruction, and the corresponding second calculation module is a matrix multiplication calculation module. Or the first instruction and the second instruction may also be other types of calculation instructions in the AI ​​chip, and the first calculation module and the second calculation module are specific calculation modules of the corresponding type, and the present application does not make specific limitations on this.

[0086] Optionally, the AI ​​chip may include, but is not limited to, a graphics processing unit (GPU), a field programmable gate array (FPGA), and an application specific integrated circuit (ASIC).

[0087] S520: Determine at least one first scalar used in the first instruction and at least one second scalar used in the second instruction.

[0088] For example, in step S220, the compiler can determine which scalars are related to the calculation results of the first instruction and the second instruction respectively. Specifically, taking the first instruction as an example, the scalar related to the calculation result of the first instruction may include the scalar appearing in the first instruction, and may also include a scalar that has a dependency relationship with the scalar appearing in the first instruction. All of the above scalars can be referred to as the first scalar used by the first instruction. For example, the compiler can traverse each scalar calculation instruction in the operator to be compiled, analyze which scalars are involved in each scalar calculation instruction, and establish a dependency table between scalars. After the dependency table is established, the compiler queries the dependency relationship of these scalars in the dependency table according to the scalars appearing in the first instruction, thereby establishing a dependency relationship between the first instruction and the scalar, and determining the scalar that has a dependency relationship with the first instruction as the first scalar. By analogy, the compiler can also determine the second scalar related to the calculation result of the second instruction according to the above method, which will not be repeated here.

[0089] S530: Determine at least one third instruction and at least one fourth instruction in the operator to be compiled.

[0090] For example, in step S530, the compiler may determine at least one third instruction that last assigns a value to each of the at least one first scalar before the execution order of the first instruction, and at least one fourth instruction that last assigns a value to each of the at least one second scalar before the execution order of the second instruction. Specifically, the instruction that last assigns a value to a scalar having a dependency relationship with the specific computing instruction before the execution of the specific computing instruction refers to one or more instructions that can determine the specific value of the scalar used in the specific computing instruction. After the compiler determines at least one third instruction and at least one fourth instruction, it assigns a label related to the execution result of the first instruction to each of the at least one third instruction, and assigns a label related to the execution result of the second instruction to each of the at least one fourth instruction, thereby facilitating the subsequent removal of redundant scalar computing instructions.

[0091] For example, take the vector calculation instruction Vector(reg_1) as an example, reg_1 is the scalar directly used in Vector(reg_1). The last scalar calculation instruction including reg_1 executed before Vector(reg_1) is executed may be an instruction that directly assigns a value to reg_1, for example, reg_1=2, then reg_1=2 may be one of the instructions that last assigned a value to scalar reg_1 before the execution sequence of Vector(reg_1). The last scalar calculation instruction including reg_1 executed before Vector(reg_1) is executed may also be a dependency relationship between reg_1 and other scalars, for example, reg_1=reg_2+1, then this instruction and the instruction that assigned a value to reg_2 before the execution sequence of this instruction can all belong to the instruction that last assigned a value to scalar reg_1 before the execution sequence of Vector(reg_1). By analogy, if there is still a dependency relationship between reg_2 and other scalars in the scalar operation instruction that last appears before the execution sequence of reg_1=reg_2+1, then continue to determine the assignment instructions for these scalars with dependencies.

[0092] It should be understood that the above example is only for the case where a specific computing instruction contains only one directly used scalar. When there are multiple directly used scalars in the specific computing instruction, the instruction for the last assignment of each scalar before the specific computing instruction is executed can be determined separately according to the above method.

[0093] For the above example, even if there is only one scalar directly used in a specific calculation instruction, and the scalar calculation instruction that last assigned a value to the scalar before the execution order of the specific calculation instruction does not have a dependency relationship with other scalars, since the specific calculation instruction may be located in different logic statements, there may be more than one instruction that last assigned a value to the scalar before the execution of the specific calculation instruction. For ease of understanding, the following takes the matrix calculation instruction Cube(reg) and the scalar reg directly used therein as an example to illustrate the situation of different logic statements.

[0094] In some possible implementations, Cube(reg) is not a loop statement, that is, Cube(reg) is not located in statements such as for(){…} or while(){…}. In this case, the order in which Cube(reg) is displayed in the IR matches its actual execution order. In other words, in this case, the statements above Cube(reg) in the IR are all statements whose execution order is before Cube(reg), and the statements below Cube(reg) are all statements whose execution order is after Cube(reg). Therefore, in this case, the compiler only needs to query the instructions in the IR of the operator to be compiled from the position of Cube(reg) one by one, and determine at least one instruction that assigns a value to reg that appears for the first time on the query path. Specifically, since Cube(reg) and the instruction that assigns a value to reg found during the upward traversal may have a cross-conditional statement, that is, Cube(reg) or the instruction that assigns a value to reg may have different execution conditions, it may be necessary to select multiple instructions that assign a value to reg to determine the last instruction that assigns a value to reg before the execution order of Cube(reg). The following is an example of different scenarios with specific examples.

[0095] Scenario 1:

[0096] reg=1;

[0097] Cube(reg);

[0098] or,

[0099] if(…){

[0100] reg=1;

[0101] Cube(reg);

[0102] }

[0103] In this scenario, Cube(reg) and the first reg assignment instruction reg=1 found from Cube(reg) upwards are not located in a conditional statement, or are located under the same condition of the same statement, that is, Cube(reg) and reg=1 do not cross conditional statements. Before the execution of Cube(reg), the assignment calculation of reg=1 will eventually be executed once under any condition, so the instruction reg=1 can be used as the last instruction to assign a value to reg before the execution of Cube(reg).

[0104] Scenario 2:

[0105] reg=0;

[0106] if(…){

[0107] reg=1;

[0108] }else{

[0109] reg=2;

[0110] }

[0111] Cube(reg);

[0112] In this scenario, Cube(reg) is outside the conditional statement, but the first instruction that assigns a value to reg that appears during the upward traversal is located in the conditional statement, that is, reg=2 is only executed in some cases before Cube(reg) is executed. Therefore, the instruction reg=2 is not enough to cover all cases before Cube(reg) is executed, and it is necessary to continue traversing upward. After continuing to traverse upward and find reg=1, although this instruction is also only executed in some cases before Cube(reg) is executed, since the if and else conditional statements cover all cases, that is, the union of the cases where reg=1 is executed and the cases where reg=2 is executed can cover all cases before Cube(reg) is executed, it is no longer necessary to continue traversing upward. In other words, in this scenario, reg=1 and reg=2 are both the last instructions to assign a value to reg before Cube(reg) is executed and need to be retained, while reg=0 is an instruction that does not affect the execution result for Cube(reg) and can be discarded.

[0113] Scenario 3:

[0114] reg=0;

[0115] if(…){

[0116] XXXX;

[0117] }else{

[0118] reg=1;

[0119] }

[0120] Cube(reg);

[0121] In this scenario, Cube(reg) is also outside the conditional statement, and the instruction reg=1 that first appears in its upward traversal and assigns a value to reg is also in the conditional statement. However, the difference from scenario 2 is that in scenario 3, there is not an instruction to assign a value to reg in all cases in the conditional statement, that is, the union of all the conditions for assigning a value to reg in the conditional statement cannot completely cover all the situations before the execution of Cube(reg). Therefore, in addition to the statement reg=1 that assigns a value to reg in the conditional statement, it is necessary to continue to search for instructions to assign a value to reg above the conditional statement. Although reg=0 outside the conditional statement will be replaced by reg=1 executed later in some cases after execution, reg=0 is actually the last instruction to assign a value to reg before Cube(reg) is executed when reg=1 is not executed. Therefore, in this scenario, reg=0 also belongs to the last instruction to assign a value to reg before Cube(reg) is executed. In other words, in this scenario, reg=0 and reg=1 are both the last instructions to assign a value to reg before Cube(reg) is executed and need to be retained.

[0122] Scenario 4:

[0123] reg=0;

[0124] if(…){

[0125] reg=1;

[0126] }else{

[0127] Cube(reg);

[0128] }

[0129] In this scenario, Cube(reg) is within the conditional statement. When traversing upward, although the first instruction that appears is reg=1, reg=1 and Cube(reg) are located under different conditions of the same conditional statement, that is, reg=1 will not be executed in any case of executing Cube(reg). Therefore, reg=1 does not belong to the last instruction that assigned a value to reg before Cube(reg) is executed. Therefore, it is necessary to continue traversing upward until a statement reg=0 that covers all situations before Cube(reg) is found outside the conditional statement. In other words, in this scenario, reg=0 is the last instruction that assigned a value to reg before Cube(reg) is executed and needs to be retained, while reg=1 is an instruction that does not affect the execution result for Cube(reg) and can be discarded.

[0130] It should be understood that the above four scenarios are only the simplest models extracted for the sake of clarity. The actual situation may be a combination of multiple scenarios, but it is only necessary to filter according to the filtering logic of each scenario. Similarly, when reg still has a dependency relationship with other scalars, the instructions for scalar assignment that has a dependency relationship with reg can also be determined based on the above four scenarios.

[0131] Through the above scheme, when the current instruction is outside the loop statement, by traversing upward from the current statement according to the arrangement order of the instruction statements in the operator, it is possible to find the instruction for the last assignment of the current scalar that does not cross the conditional statement with the current instruction, and only retain these found instructions in the scalar assignment instructions of the compiled operator, discarding redundant calculations that do not affect the calculation results of the current instruction, thereby improving the calculation efficiency of the compiled operator.

[0132] In some possible implementations, Cube(reg) is not a loop statement, that is, Cube(reg) is not in a for(){...} or while(){...} statement. In this case, since Cube(reg) needs to be executed multiple times, the statements in the IR that are below Cube(reg) but in the same loop statement as Cube(reg) are before the next execution of Cube(reg) in the execution order, so the instructions that last assigned a value to reg before the execution of Cube(reg) also need to be taken into account.

[0133] Specifically, the compiler first follows the search logic when Cube(reg) is not a loop statement, and traverses upward from the location of Cube(reg) to find statements that affect the execution result of Cube(reg). There are two types of situations for the at least one instruction that assigns a value to reg for the last time found by traversing upward. For the sake of description, the at least one instruction that assigns a value to reg for the last time is referred to as the at least one eighth instruction.

[0134] The first case is that each of at least one of the eighth instructions belongs to the same level of the same loop statement as Cube(reg). This case means that at least one of the eighth instructions must be executed before Cube(reg) is executed in each loop. In other words, reg will be assigned a value before Cube(reg) is executed in each loop. In this case, at least one of the eighth instructions is the last instruction to assign a value to reg before Cube(reg) is executed, so there is no need to continue traversing other instructions for searching. For example:

[0135] reg=0;

[0136] for(…){

[0137] if(…){

[0138] reg=1;

[0139] }else{

[0140] reg=2;

[0141] }

[0142] Cube(reg);

[0143] reg=3;

[0144] …

[0145] }

[0146] In the above example, the instructions for assigning a value to reg found by traversing upward from Cube(reg) according to the search logic described above are reg=1 and reg=2. Since the union of the execution conditions of these two instructions has covered all situations before the execution of Cube(reg), it will not continue to query upward to reg=0. Since reg=1 and reg=2 are in the same for() loop as Cube(reg), the value of reg can only be 1 or 2 each time Cube(reg) is executed in the loop, so there is no need to traverse and search for other instructions. In other words, in this scenario, reg=1 and reg=2 are the last instructions for assigning a value to reg before the execution of Cube(reg) and need to be retained, while reg=0 and reg=3 are instructions that do not affect the execution result for Cube(reg) and can be discarded.

[0147] The second type of situation is that at least one of the eighth instructions contains instructions that are not at the same level as the same loop statement as Cube(reg). This type of situation means that at least one of the eighth instructions may not be executed before Cube(reg) is executed in each loop. In other words, reg may not be assigned a value again before Cube(reg) is executed in each loop. In this case, at least one eighth instruction is not necessarily the last instruction to assign a value to reg before Cube(reg) is executed, so it is necessary to traverse downward in the loop statements at the same level of Cube(reg). Specifically, from the position of Cube(reg), query one by one the instructions at the same level in the same loop statement as Cube(reg), and determine at least one instruction that last appeared to assign a value to reg during the query process. For the sake of description, it is referred to as at least one ninth instruction. The compiler jointly determines the aforementioned at least one eighth instruction and at least one ninth instruction as the last instruction to assign a value to reg before Cube(reg) is executed. For example:

[0148] reg=0;

[0149] for(…){

[0150] if(…){

[0151] reg=1;

[0152] }

[0153] Cube(reg);

[0154] …

[0155] reg=2;

[0156] }

[0157] In the above example, the instructions for assigning a value to reg found by traversing upward from Cube(reg) according to the search logic described above are reg=1 and reg=2, where reg=1 is located at the same level of the same loop statement of Cube(reg). Since the execution condition of reg=1 cannot cover all situations before the execution of Cube(reg), and reg=0 and Cube(reg) do not belong to the same loop statement level, there may be a situation in the loop process where reg=1 and reg=2 are not executed before a certain execution of Cube(reg). In this case, since other instructions at the same level of the same loop statement as Cube(reg) were executed before the current execution of Cube(reg) and before the last execution of Cube(reg), it is necessary to consider whether these instructions have changed the value of reg. Therefore, traverse downward from Cube(reg) to find the end of the loop statement at the same level, the statement reg=2 that last assigned a value to reg in a loop. If the execution condition of the if() conditional statement is not met in a certain loop, that is, the reg=1 instruction is not executed before the execution of Cube(reg), then the statement that takes effect on the assignment of reg during the execution of Cube(reg) is the last reg=2 executed in the previous loop. In other words, in this scenario, reg=0, reg=1 and reg=2 are all the last instructions for assigning a value to reg before the execution of Cube(reg), and all need to be retained.

[0158] It should be understood that the above example is only an example of a loop statement. When Cube(reg) is in a multi-layer loop statement, when traversing downward to determine at least one ninth instruction, the last instruction to assign a value to reg in the loop of each level can be determined in each level according to the above method until the entire loop statement ends or reaches the same loop level as the outermost eighth statement of the loop level where at least one eighth instruction is located.

[0159] It should be understood that in the process of determining at least one eighth statement and / or at least one ninth statement, if there is a situation where Cube(reg) and the statement that assigns a value to reg cross conditional statements, the appropriate number of eighth statements and / or ninth statements can be determined according to the various scenarios listed above when Cube(reg) is not in the loop statement, and no further details are given here. Similarly, when Cube(reg) has multiple scalars with dependencies, the instructions for assigning values ​​to each scalar can be determined separately according to the above logic.

[0160] Through the above scheme, when the current instruction is located in a loop statement, in addition to traversing upward from the current statement according to the arrangement order of the instruction statements in the operator, it also traverses downward within the loop statement to which the current statement belongs. Through the characteristics of the cyclic execution of the loop statement, it is possible to find the instruction that is actually the last assignment of the current scalar in the execution order, and only retain these found instructions in the scalar assignment instructions of the compiled operator, discarding redundant calculations that do not affect the calculation results of the current instruction, thereby improving the calculation efficiency of the compiled operator.

[0161] S540: Generate a first target operator from the first instruction and at least one third instruction, and generate a second target operator from the second instruction and at least one fourth instruction.

[0162] For example, in step S540, the compiler may generate a first target operator executed by a first computing module and a second target operator executed by a second computing module in a loosely coupled architecture AI chip. Specifically, the compiler may retain, in the IR of the first target operator, only the first instruction that needs to be performed by the first computing module to perform a specific type of calculation, at least one third instruction assigned a label related to the execution result of the first instruction, and control logic instructions related to the above instructions, based on the label assigned to each scalar calculation statement. Similarly, the compiler may retain in the IR of the second target operator only the second instruction that needs to be performed by the second computing module to perform a specific type of calculation, at least one fourth instruction assigned a label related to the execution result of the second instruction, and control logic instructions related to the above instructions.

[0163] Through the technical solution of the embodiment of the present application, a dependency relationship between instructions and scalars executed by a specific computing module is established, and the scalars used in each instruction executed by the specific computing module, as well as the instructions that have an actual impact on the assignment of these scalars before being used by these instructions, can be determined. This allows the operator compiled by the specific computing module to eliminate redundant instructions that have no impact on the calculation results, thereby improving the computational efficiency of the compiled operator.

[0164] The above solution can solve the problem that when the tightly coupled architecture is compatible with the loosely coupled architecture, there are too many redundant computing instructions in the compiled operator, which reduces the computing efficiency of the compiled operator. However, as mentioned above, when the tightly coupled architecture is compatible with the loosely coupled architecture, there may be a problem that the original data path between the computing modules disappears. Therefore, it is necessary to further solve the problem of data transfer between different types of computing modules in the loosely coupled architecture.

[0165] Therefore, for Figure 2 In the operator compilation method shown, in some possible implementations, the method also includes: determining a fifth instruction from the operator to be compiled, the fifth instruction being used to instruct the interaction data to be transmitted between the first cache area of ​​the first computing module and the second cache area of ​​the second computing module; determining a sixth instruction and a seventh instruction based on the fifth instruction, the sixth instruction being used to instruct the interaction data to be written into the global memory of the artificial intelligence chip, and the seventh instruction being used to instruct the interaction data to be read from the global memory; adding the sixth instruction to the first target operator and adding the seventh instruction to the second target operator, and / or, adding the sixth instruction to the second target operator and adding the seventh instruction to the first target operator.

[0166] Specifically, the compiler can identify specific instruction content, thereby determining the instruction used to instruct two computing modules to directly exchange data through the cache area in the tightly coupled architecture. For example, Copy(v_buf0, c_buf0) and Copy(v_buf1, c_buf1) in the above example both belong to the fifth instruction, Copy(v_buf0, c_buf0) instructs the cache area c_buf0 in the first computing module and the cache area v_buf0 in the second computing module to exchange data, and Copy(v_buf1, c_buf1) instructs the cache area c_buf1 in the first computing module and the cache area v_buf1 in the second computing module to exchange data. Since the data path between the first computing module and the second computing module does not exist under the loosely coupled architecture, the above process needs to use the in-core memory as a transfer, and the instruction of the next data interaction in the original architecture is split into two: the sixth instruction of gm executed by one computing module to write data into the in-core memory, and the seventh instruction of gm executed by another computing module to read data from the in-core memory. Taking the Copy(v_buf0,c_buf0) instruction as an example, the instruction at the corresponding position in the target operator IR of the compiled first computing module can be Copy(workspace0,c_buf0), which is used to indicate that the data in the cache area c_buf0 is written to the workspace workspace0 of gm; the instruction at the corresponding position in the target operator IR of the compiled second computing module is Copy(v_buf0,workspace0), which is used to indicate that the data in the workspace workspace0 of gm is read to the cache area v_buf0.

[0167] Through the technical solution of the embodiment of the present application, the global memory in the chip transfers data that was originally exchanged directly through their respective cache areas between two different types of computing modules under a tightly coupled architecture, which can solve the problem of non-existence of some data paths caused by the adjustment of the hardware architecture from tight coupling to loose coupling, thereby improving the scope of application of the compiled operator.

[0168] For the above scheme, the data transfer uses the workspace of gm, but the specific allocation of which part and how much workspace capacity is used for data transfer is allocated by gm, so there may be a problem of excessive memory usage in the kernel. For example, when the Copy(v_buf0,c_buf0) instruction in the original operator is located in a loop statement, for the corresponding Copy(workspace0,c_buf0) instruction in the compiled target operator, workspace0 will be requested from gm once each time the loop is executed. In this case, gm may need to divide a part of the physical memory space and redefine it as workspace0 each time, and the physical memory space used as workspace0 in the previous loop may not be released in time, resulting in repeated occupation of the workspace. For another example, for a set of instructions Copy(v_buf0,c_buf0) and Copy(c_buf0,v_buf0) for a set of data exchange in the original operator, the required buffer size is actually the same as the buffer size required when executing one of them for unidirectional data transfer, and the above process after compilation requires two workspaces for gm.

[0169] Therefore, in some possible implementations, before determining the sixth instruction and the seventh instruction according to the fifth instruction, the method also includes: establishing a mirror area in the working space of the global memory, the mirror area being a mapping of the cache area with the smallest capacity between the first cache area and the second cache area; determining the sixth instruction and the seventh instruction according to the fifth instruction, including: determining the sixth instruction and the seventh instruction according to the fifth instruction and the mirror area, the sixth instruction being used to instruct writing interactive data into the mirror area, and the seventh instruction being used to instruct reading interactive data from the mirror area.

[0170] Specifically, Figure 6 FIG. 1 shows a schematic diagram of the change in the data transfer process when a tightly coupled architecture is compatible with a loosely coupled architecture. Figure 6As shown in the figure, under the tightly coupled architecture, the matrix multiplication calculation module (cube core) and the vector calculation module (vector core) have a directly connected data path, so they can directly exchange data in the corresponding buffers. For example, the buffer 0 (c_buf0) in the cube core and the buffer 0 (v_buf0) in the vector core directly exchange data, and the buffer 1 (c_buf1) in the cube core and the buffer 1 (v_buf1) in the vector core directly exchange data. In the figure, the size of the buffer rectangle represents the size of the buffer capacity. Since the capacity of each buffer is fixed, no matter how much and how many times the cube core and the vector core exchange data through c_buf0 and v_buf0, the maximum amount of data exchanged in a single data exchange is restricted by the capacity of the smaller buffer in c_buf0 and v_buf0. Similarly, the maximum amount of data exchanged in a single data exchange between c_buf1 and v_buf1 is also restricted by the capacity of this smaller buffer. The above characteristics mean that the physical memory space required for data transfer does not need to consider specific data transfer instructions, but can be determined based on the size of the cache area in the computing module.

[0171] like Figure 6 As shown in the figure, under the loosely coupled architecture, the cube core and the vector core rely on the workspace of gm for data transfer. According to the above ideas, a fixed-size mirror area can be established in the workspace to transfer data between computing modules. For example, the data between c_buf0 and v_buf0 is transferred through the mirror area 0 in the workspace. Since the capacity of c_buf0 is smaller than the capacity of v_buf0, the capacity of mirror area 0 is determined according to the capacity of c_buf0. And mirror area 0 is the mapping of c_buf0 in the workspace, so the cube core can keep the states of mirror area 0 and c_buf0 consistent through instructions, so that v_buf0 can interact with mirror area 0 according to the logic of interacting with c_buf0 data in the original operator. Similarly, the data between c_buf1 and v_buf1 is transferred through mirror area 1 in the workspace. Since the capacity of c_buf1 is larger than that of v_buf1, the capacity of mirror area 1 is determined according to the capacity of v_buf1 and is the mapping of v_buf1 in the workspace.

[0172] Through the technical solution of the embodiment of the present application, a mirror area of ​​the cache area in the computing module is established in the working space of the global memory, and the capacity of the mirror area is determined according to the capacity of the cache area, so that only a fixed size of capacity in the global memory is needed to meet the needs of data transfer between two different computing modules, avoiding the need to re-apply for space in the global memory for each data transfer, thereby saving the memory space of the AI ​​chip occupied by the compiled operator during calculation.

[0173] Optionally, the mirror area in the workspace can also be used to transfer scalar values ​​that have dependencies with the first instruction and the second instruction. For example, the instruction in the IR of the original operator can change the value of the scalar reg through cube(reg), and then the scalar reg is used when executing the vector(reg) instruction. In the compiled operator, since cube(reg) and vector(reg) are executed separately in two target operators, the scalar reg needs to be transferred.

[0174] In the above case, the method also includes: determining at least one third scalar according to the first instruction and the second instruction, the at least one third scalar being a scalar included in both the at least one first scalar and the at least one second scalar; establishing a mirror area in the working space of the global memory, including: establishing the mirror area according to the size of the first cache area, the size of the second cache area, and the size of each third scalar in the at least one third scalar, wherein the size of the mirror area is the sum of the size of the cache area with the smallest capacity among the first cache area and the second cache area and the size of each third scalar in the at least one third scalar.

[0175] Through the technical solution of the embodiment of the present application, a mirror area is established in the working space of the global memory for the transfer of associated scalars in instructions executed by different computing modules, which can realize synchronous computing between heterogeneous cores, thereby improving the applicability of the compiled operator.

[0176] The following example shows the target operator obtained by the method of compiling operators provided in this application. The operator IR to be compiled is the same as that in the previous example:

[0177]

[0178] Through the method of compiling operators provided in this application, the target operator IR executed by cube core is obtained:

[0179]

[0180] By using the operator compilation method provided in this application, the target operator IR executed by the vector core is obtained:

[0181]

[0182] Among them, workspace[0] represents the mirror area 0 established in the workspace, and workspace[x] represents the mirror area 1 established in the workspace. It can be seen that the target operator obtained by the method of compiling operators provided in this application can reduce redundant scalar calculation instructions, improve the calculation efficiency of the compiled operator, and also save the memory space occupied by the compiled operator in the AI ​​chip during calculation.

[0183] Combination of the above Figures 5 and 6 The method embodiment of compiling an operator provided by the present application is described below. Figures 7 and 8 , an embodiment of the device for compiling the operator provided in this application is described.

[0184] Figure 7 A schematic structural block diagram of a device 700 for compiling an operator provided in an embodiment of the present application is shown.

[0185] like Figure 7 As shown, the device 700 includes: an acquisition module 710 , a determination module 720 , an identification module 730 , and a generation module 740 .

[0186] Specifically, the acquisition module 710 is used to acquire operators to be compiled, which include a first instruction executed by a first computing module in the artificial intelligence chip and a second instruction executed by a second computing module in the artificial intelligence chip. The types of computing tasks performed by the first computing module and the second computing module are different.

[0187] Specifically, the determination module 720 is used to determine at least one first scalar used in the first instruction and at least one second scalar used in the second instruction.

[0188] Specifically, the identification module 730 is used to determine at least one third instruction and at least one fourth instruction in the operator to be compiled, wherein the at least one third instruction is an instruction that last assigns a value to each first scalar in at least one first scalar before the execution order of the first instruction, and the at least one fourth instruction is an instruction that last assigns a value to each second scalar in at least one second scalar before the execution order of the second instruction.

[0189] Optionally, the identification module 730 is specifically used to query the instructions in the operator to be compiled one by one from the current instruction position upward when the current instruction is not a loop statement, wherein the current instruction is the first instruction or the second instruction; determine at least one eighth instruction, wherein the at least one eighth instruction is an instruction that assigns a value to the current scalar and appears for the first time in the query path under each first condition in at least one first condition, and the at least one first condition is a condition for the execution of the current instruction, wherein the current scalar is at least one first scalar used in the first instruction or at least one second scalar used in the second instruction; determine at least one eighth instruction as a target instruction, wherein the target instruction is at least one third instruction corresponding to the first instruction or at least one fourth instruction corresponding to the second instruction.

[0190] Optionally, the identification module 730 is specifically used to, when the current instruction is a loop statement, query the instructions in the operator to be compiled one by one from the current instruction position upward, wherein the current instruction is the first instruction or the second instruction; determine at least one eighth instruction, wherein the at least one eighth instruction is the instruction that assigns a value to the current scalar that appears for the first time during the query process under each first condition in at least one first condition, and the at least one first condition is the condition for the execution of the current instruction, wherein the current scalar is at least one first scalar used in the first instruction or at least one second scalar used in the second instruction; when each eighth instruction in at least one eighth instruction belongs to a loop statement at the same level as the current instruction, determine at least one eighth instruction as a target instruction, wherein the target instruction is at least one third instruction corresponding to the first instruction or at least one fourth instruction corresponding to the second instruction; when at least one eighth instruction does not belong to a loop statement at the same level as the current instruction, query the loop statements that belong to the same level as the current instruction from the current instruction position downward one by one; determine at least one ninth instruction, wherein the at least one eighth instruction is the instruction that assigns a value to the current scalar that appears for the last time during the query process under each first condition in at least one first condition; and determine at least one eighth instruction and at least one ninth instruction as target instructions.

[0191] Specifically, the generation module 740 is used to generate a first target operator from a first instruction and at least one third instruction, and to generate a second target operator from a second instruction and at least one fourth instruction. The instructions in the first target operator are executed by the first computing module, and the instructions in the second target operator are executed by the second computing module.

[0192] Optionally, the device 700 may also include a transfer module 750, which is used to determine a fifth instruction from the operator to be compiled, the fifth instruction being used to instruct the interaction data to be transmitted between the first cache area of ​​the first computing module and the second cache area of ​​the second computing module; determine a sixth instruction and a seventh instruction based on the fifth instruction, the sixth instruction being used to instruct the interaction data to be written into the global memory of the artificial intelligence chip, and the seventh instruction being used to instruct the interaction data to be read from the global memory; add the sixth instruction to the first target operator and add the seventh instruction to the second target operator, and / or add the sixth instruction to the second target operator and add the seventh instruction to the first target operator.

[0193] Optionally, the device 700 may further include a mapping module 760, which establishes a mirror area in the working space of the global memory before determining the sixth instruction and the seventh instruction according to the fifth instruction, and the mirror area is a mapping of the cache area with the smallest capacity between the first cache area and the second cache area; a transfer module, which is used to: determine the sixth instruction and the seventh instruction according to the fifth instruction and the mirror area, the sixth instruction is used to instruct to write the interactive data into the mirror area, and the seventh instruction is used to instruct to read the interactive data from the mirror area.

[0194] Optionally, the transfer module 750 is further used to determine at least one third scalar according to the first instruction and the second instruction, and the at least one third scalar is a scalar included in both the at least one first scalar and the at least one second scalar; the mapping module 760 is specifically used to establish a mirror area according to the size of the first cache area, the size of the second cache area, and the size of each third scalar in the at least one third scalar, wherein the size of the mirror area is the sum of the size of the cache area with the smallest capacity in the first cache area and the second cache area and the size of each third scalar in the at least one third scalar.

[0195] Optionally, the operator to be compiled may be an operator of an artificial intelligence (AI) chip suitable for a tightly coupled architecture, and the target operator may be an operator of an AI chip suitable for a loosely coupled architecture.

[0196] Optionally, the AI ​​chip may include, but is not limited to, a graphics processing unit (GPU), a field programmable gate array (FPGA), and an application specific integrated circuit (ASIC).

[0197] Optionally, the first calculation module may be a matrix multiplication calculation module, and the second calculation module may be a vector calculation module, or the first calculation module may be a vector calculation module, and the second calculation module may be a matrix multiplication calculation module.

[0198] Among them, the above modules can be implemented by software or hardware. Exemplarily, the implementation of the identification module 730 is introduced below by taking the identification module 730 as an example. Similarly, the implementation of the acquisition module 710, the determination module 720, the generation module 740, the transfer module 750 and the mapping module 760 can refer to the implementation of the identification module 730.

[0199] As an example of a software functional unit, the identification module 730 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the identification module 730 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

[0200] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0201] As an example of a hardware functional unit, the identification module 730 may include at least one computing device, such as a server, etc. Alternatively, the identification module 730 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0202] The multiple computing devices included in the identification module 730 can be distributed in the same region or in different regions. The multiple computing devices included in the identification module 730 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the identification module 730 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0203] It should be noted that, in other embodiments, the acquisition module 710, the determination module 720, the identification module 730, the generation module 740, the transfer module 750 and the mapping module 760 can be used to execute any step in the above-mentioned method of compiling the operator respectively, and the steps that the acquisition module 710, the determination module 720, the identification module 730, the generation module 740, the transfer module 750 and the mapping module 760 are responsible for implementing can be specified as needed. The acquisition module 710, the determination module 720, the identification module 730, the generation module 740, the transfer module 750 and the mapping module 760 respectively implement different steps in the above-mentioned method of compiling the operator to realize all the functions of the operation and maintenance channel management cluster.

[0204] The present application also provides a computing device 100. Figure 8 As shown, the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other through the bus 102. The computing device 100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.

[0205] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus 102 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 102 may include a path for transmitting information between various components of the computing device 100 (eg, the memory 106, the processor 104, and the communication interface 108).

[0206] The processor 104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0207] The memory 106 may include a volatile memory, such as a random access memory (RAM). The memory 106 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0208] The memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the aforementioned acquisition module, determination module, identification module, generation module, transfer module, and mapping module, thereby implementing the aforementioned method of compiling operators. That is, the memory 106 stores instructions for executing the aforementioned method of compiling operators.

[0209] The communication interface 108 uses a command distribution module such as, but not limited to, a network interface card, a transceiver, etc. to implement communication between the computing device 100 and other devices or a communication network.

[0210] An embodiment of the present application also provides a chip, which includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to execute the above-mentioned method of compiling an operator.

[0211] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the above-mentioned method of compiling an operator.

[0212] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by the computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state hard disk), etc. The computer-readable storage medium includes instructions, and the instructions instruct the computing device to perform the above-mentioned management. The various technical features can be combined arbitrarily. In order to make the description concise, all possible combinations of the various technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0213] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for compiling an operator, characterized in that: include: Obtain an operator to be compiled, wherein the operator to be compiled includes a first instruction executed by a first computing module in the artificial intelligence chip and a second instruction executed by a second computing module in the artificial intelligence chip, and the first computing module and the second computing module execute different types of computing tasks; determining at least one first scalar used in the first instruction and at least one second scalar used in the second instruction; Determine at least one third instruction and at least one fourth instruction in the operator to be compiled, wherein the at least one third instruction is an instruction that last assigns a value to each first scalar in the at least one first scalar before the execution order of the first instruction, and the at least one fourth instruction is an instruction that last assigns a value to each second scalar in the at least one second scalar before the execution order of the second instruction; A first target operator is generated by the first instruction and the at least one third instruction, a second target operator is generated by the second instruction and the at least one fourth instruction, the instructions in the first target operator are executed by the first computing module, and the instructions in the second target operator are executed by the second computing module.

2. The method according to claim 1, characterized in that The method further comprises: Determine a fifth instruction from the operator to be compiled, where the fifth instruction is used to instruct the interactive data to be transmitted between the first buffer area of ​​the first computing module and the second buffer area of ​​the second computing module; Determine a sixth instruction and a seventh instruction according to the fifth instruction, wherein the sixth instruction is used to instruct writing the interaction data into the global memory of the artificial intelligence chip, and the seventh instruction is used to instruct reading the interaction data from the global memory; Adding the sixth instruction to the first target operator and adding the seventh instruction to the second target operator, and / or adding the sixth instruction to the second target operator and adding the seventh instruction to the first target operator.

3. The method according to claim 2, characterized in that Before determining the sixth instruction and the seventh instruction according to the fifth instruction, the method further includes: Establishing a mirror area in the working space of the global memory, wherein the mirror area is a mapping of the cache area with the smallest capacity between the first cache area and the second cache area; Determining the sixth instruction and the seventh instruction according to the fifth instruction includes: A sixth instruction and a seventh instruction are determined according to the fifth instruction and the mirror area, the sixth instruction is used to instruct writing the interactive data into the mirror area, and the seventh instruction is used to instruct reading the interactive data from the mirror area.

4. The method according to claim 2 or 3, characterized in that: The method further comprises: Determine at least one third scalar according to the first instruction and the second instruction, wherein the at least one third scalar is a scalar included in both the at least one first scalar and the at least one second scalar; The step of establishing a mirror area in the working space of the global memory includes: The mirror area is established according to the size of the first cache area, the size of the second cache area, and the size of each third scalar in the at least one third scalar, wherein the size of the mirror area is the sum of the size of the cache area with the smallest capacity between the first cache area and the second cache area and the size of each third scalar in the at least one third scalar.

5. The method according to any one of claims 1 to 4, characterized in that: The determining at least one third instruction and at least one fourth instruction in the operator to be compiled includes: When the current instruction is not a loop statement, query the instructions in the operator to be compiled one by one from the current instruction position upward, wherein the current instruction is the first instruction or the second instruction; determining at least one eighth instruction, the at least one eighth instruction being an instruction for assigning a value to a current scalar that appears for the first time in a query path under each first condition in at least one first condition, the at least one first condition being a condition for execution of the current instruction, wherein the current scalar is the at least one first scalar used in the first instruction or the at least one second scalar used in the second instruction; The at least one eighth instruction is determined as a target instruction, wherein the target instruction is the at least one third instruction corresponding to the first instruction or the at least one fourth instruction corresponding to the second instruction.

6. The method according to any one of claims 1 to 4, characterized in that The determining at least one third instruction and at least one fourth instruction in the operator to be compiled includes: When the current instruction is a loop statement, query the instructions in the operator to be compiled one by one from the current instruction position upward, wherein the current instruction is the first instruction or the second instruction; determining at least one eighth instruction, the at least one eighth instruction being an instruction for assigning a value to a current scalar that appears for the first time during a query in each of at least one first condition, the at least one first condition being a condition for executing the current instruction, wherein the current scalar is the at least one first scalar used in the first instruction or the at least one second scalar used in the second instruction; In a case where each of the at least one eighth instruction and the current instruction belong to a loop statement at the same level, determining the at least one eighth instruction as a target instruction, wherein the target instruction is the at least one third instruction corresponding to the first instruction or the at least one fourth instruction corresponding to the second instruction; In the case that the at least one eighth instruction and the current instruction are not loop statements at the same level, searching for loop statements at the same level as the current instruction one by one from the current instruction position downward; determining at least one ninth instruction, wherein the at least one eighth instruction is an instruction for assigning a value to the current scalar that appears last in the query process under each first condition of the at least one first condition; The at least one eighth instruction and the at least one ninth instruction are determined as the target instructions.

7. The method according to any one of claims 1 to 6, characterized in that The artificial intelligence chip is a graphics processor, a field programmable gate array or a dedicated integrated circuit.

8. The method according to any one of claims 1 to 7, characterized in that The first calculation module is a matrix multiplication calculation module, and / or the second calculation module is a vector calculation module.

9. A device for compiling an operator, characterized in that: include: an acquisition module, configured to acquire an operator to be compiled, wherein the operator to be compiled includes a first instruction executed by a first computing module in the artificial intelligence chip and a second instruction executed by a second computing module in the artificial intelligence chip, and the first computing module and the second computing module execute different types of computing tasks; a determination module, configured to determine at least one first scalar used in the first instruction and at least one second scalar used in the second instruction; an identification module, configured to determine at least one third instruction and at least one fourth instruction in the operator to be compiled, wherein the at least one third instruction is an instruction that last assigns a value to each first scalar in the at least one first scalar before the execution order of the first instruction, and the at least one fourth instruction is an instruction that last assigns a value to each second scalar in the at least one second scalar before the execution order of the second instruction; A generation module is used to generate a first target operator from the first instruction and the at least one third instruction, and to generate a second target operator from the second instruction and at least one fourth instruction, wherein the instructions in the first target operator are executed by the first computing module, and the instructions in the second target operator are executed by the second computing module.

10. The device according to claim 9, characterized in that The device also includes a transport module for: Determine a fifth instruction from the operator to be compiled, where the fifth instruction is used to instruct the interactive data to be transmitted between the first buffer area of ​​the first computing module and the second buffer area of ​​the second computing module; Determine a sixth instruction and a seventh instruction according to the fifth instruction, wherein the sixth instruction is used to instruct writing the interaction data into the global memory of the artificial intelligence chip, and the seventh instruction is used to instruct reading the interaction data from the global memory; Adding the sixth instruction to the first target operator and adding the seventh instruction to the second target operator, and / or adding the sixth instruction to the second target operator and adding the seventh instruction to the first target operator.

11. The device according to claim 10, characterized in that The apparatus further comprises a mapping module, configured to: Before determining the sixth instruction and the seventh instruction according to the fifth instruction, establishing a mirror area in the working space of the global memory, the mirror area being a mapping of the cache area with the smallest capacity between the first cache area and the second cache area; The transport module is used for: A sixth instruction and a seventh instruction are determined according to the fifth instruction and the mirror area, the sixth instruction is used to instruct writing the interactive data into the mirror area, and the seventh instruction is used to instruct reading the interactive data from the mirror area.

12. The method according to claim 10 or 11, characterized in that: The transport module is also used for: Determine at least one third scalar according to the first instruction and the second instruction, wherein the at least one third scalar is a scalar included in both the at least one first scalar and the at least one second scalar; The mapping module is used to: The mirror area is established according to the size of the first cache area, the size of the second cache area, and the size of each third scalar in the at least one third scalar, wherein the size of the mirror area is the sum of the size of the cache area with the smallest capacity between the first cache area and the second cache area and the size of each third scalar in the at least one third scalar.

13. The device according to any one of claims 9 to 12, characterized in that The identification module is used for: When the current instruction is not a loop statement, query the instructions in the operator to be compiled one by one from the current instruction position upward, wherein the current instruction is the first instruction or the second instruction; determining at least one eighth instruction, the at least one eighth instruction being an instruction for assigning a value to a current scalar that appears for the first time in a query path under each first condition in at least one first condition, the at least one first condition being a condition for execution of the current instruction, wherein the current scalar is the at least one first scalar used in the first instruction or the at least one second scalar used in the second instruction; The at least one eighth instruction is determined as a target instruction, wherein the target instruction is the at least one third instruction corresponding to the first instruction or the at least one fourth instruction corresponding to the second instruction.

14. The device according to any one of claims 9 to 12, characterized in that The identification module is used to: When the current instruction is a loop statement, query the instructions in the operator to be compiled one by one from the current instruction position upward, wherein the current instruction is the first instruction or the second instruction; determining at least one eighth instruction, the at least one eighth instruction being an instruction for assigning a value to a current scalar that appears for the first time during a query in each of at least one first condition, the at least one first condition being a condition for executing the current instruction, wherein the current scalar is the at least one first scalar used in the first instruction or the at least one second scalar used in the second instruction; In a case where each of the at least one eighth instruction and the current instruction belong to a loop statement at the same level, determining the at least one eighth instruction as a target instruction, wherein the target instruction is the at least one third instruction corresponding to the first instruction or the at least one fourth instruction corresponding to the second instruction; In the case that the at least one eighth instruction and the current instruction are not loop statements at the same level, searching for loop statements at the same level as the current instruction one by one from the current instruction position downward; determining at least one ninth instruction, wherein the at least one eighth instruction is an instruction for assigning a value to the current scalar that appears last in the query process under each first condition of the at least one first condition; The at least one eighth instruction and the at least one ninth instruction are determined as the target instructions.

15. The device according to any one of claims 9 to 14, characterized in that The artificial intelligence chip is a graphics processor, a field programmable gate array or a dedicated integrated circuit.

16. The device according to any one of claims 9 to 15, characterized in that The first calculation module is a matrix multiplication calculation module, and / or the second calculation module is a vector calculation module.

17. A computing device, characterized in that The computer comprises a processor and a memory, wherein the processor is configured to execute instructions stored in the memory so that the computing device executes the method according to any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that: The method comprises computer program instructions which, when executed by a computing device, cause the computing device to perform the method according to any one of claims 1 to 8.

Images