Processor resource allocation method and device, equipment and medium

Abstract

The invention provides a processor resource allocation method and device, equipment and a medium. The method comprises the steps that the lifetime of a plurality of storage objects in a plurality of instructions of a program to be compiled is obtained, and the lifetime of a first storage object in the plurality of storage objects comprises an instruction sequence involved between a start instruction that the first storage object is used for the first time and an end instruction that the first storage object is used for the last time; in the survival period of the first storage object, the first storage object is in a survival state; the storage resources are allocated to the storage objects in the living state in the multiple instructions, and the allocation of the storage resources to the storage objects in the living state in a first instruction in the multiple instructions is prior to the allocation of the storage resources to the storage objects in the living state in a second instruction in the multiple instructions; the total storage amount of the storage objects in the survival state in the first instruction is greater than the total storage amount of the storage objects in the survival state in the second instruction.

Description

Technical Field

[0001] At least one embodiment of the present disclosure relates to the field of artificial intelligence, and more specifically to a method for allocating processor resources during a program compilation process, an apparatus for allocating processor resources during a program compilation process, an electronic device, and a non-transitory computer-readable storage medium. Background Art

[0002] In the field of artificial intelligence, convolutional neural networks are often used to process image and video data to extract features from images, thereby achieving effective recognition and understanding of images. Both the training and reasoning of convolutional neural networks require the use of programs involving tensor operations. Such programs are usually run using graphics processors such as general purpose computing on graphics processing units (GPGPUs) and graphics processing units (GPUs). In the compilation of such programs, it is necessary to map variables, arrays, matrices, tensors and other storage objects in the program to appropriate memory locations, that is, to allocate physical addresses of storage resources to variables, arrays, matrices, tensors and other storage objects in the program, so that these variables, arrays, matrices, tensors and other storage objects can store data at the allocated physical addresses during program execution. There are many different types of memory objects that need to be allocated physical addresses of storage resources. For example, in GPGPUs, GPUs, etc., storage resources include registers, shared memory, descriptor indexes, constant memory addresses, synchronization barriers, and so on. In traditional practices, programmers manually allocate storage resources to some storage objects, while drivers or compilers allocate storage resources to other storage objects. Summary of the invention

[0003] According to one aspect of the present disclosure, at least one embodiment provides a method for allocating processor resources, including: obtaining the lifetimes of multiple storage objects in multiple instructions of a program to be compiled, wherein the lifetime of a first storage object among the multiple storage objects includes an instruction sequence involved from a start instruction in which the first storage object is used for the first time to an end instruction in which the first storage object is used for the last time, and during the lifetime of the first storage object, the first storage object is in a survival state; allocating storage resources to the storage objects in the survival state in the multiple instructions, wherein allocating storage resources to the storage objects in the survival state in the first instruction among the multiple instructions takes precedence over allocating storage resources to the storage objects in the survival state in the second instruction among the multiple instructions, and wherein the total storage amount of the storage objects in the survival state in the first instruction is greater than the total storage amount of the storage objects in the survival state in the second instruction.

[0004] According to another aspect of the present disclosure, at least one embodiment provides a device for allocating processor resources, including: an acquisition module, configured to acquire the lifetimes of multiple storage objects in multiple instructions of a program to be compiled, wherein the lifetime of a first storage object among the multiple storage objects includes an instruction sequence involved from a start instruction in which the first storage object is used for the first time to an end instruction in which the first storage object is used for the last time, and during the lifetime of the first storage object, the first storage object is in a survival state; an allocation module, configured to allocate storage resources to the storage objects in a survival state in the multiple instructions, wherein allocating storage resources to the storage objects in a survival state in the first instruction among the multiple instructions takes precedence over allocating storage resources to the storage objects in a survival state in the second instruction among the multiple instructions, and wherein the total storage amount of the storage objects in a survival state in the first instruction is greater than the total storage amount of the storage objects in a survival state in the second instruction.

[0005] According to another aspect of the present disclosure, at least one embodiment provides a device for allocating processor resources, including: a storage device for storing computer instructions; a processing device for reading the computer instructions in the storage device and executing a method according to at least one embodiment of the present disclosure.

[0006] According to another aspect of the present disclosure, at least one embodiment provides an electronic device including a device for allocating processor resources according to at least one embodiment of the present disclosure.

[0007] According to another aspect of the present disclosure, at least one embodiment provides a non-transitory computer-readable storage medium having computer instructions stored thereon, wherein when the computer instructions are executed by a processing device, the processing device performs a method according to at least one embodiment of the present disclosure.

[0008] According to another aspect of the present disclosure, at least one embodiment provides a computer program product, including computer instructions, wherein when the computer instructions are executed by a processing device, the processing device performs a method according to at least one embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0010] Figure 1 A flowchart of a method for processor resource allocation according to at least one embodiment of the present disclosure is shown.

[0011] Figure 2A and Figure 2B A schematic diagram showing the instruction control flow of a program according to at least one embodiment of the present disclosure.

[0012] Figure 3 A schematic diagram showing the correspondence between each instruction in a program segment and a storage object in a live state according to at least one embodiment of the present disclosure.

[0013] Figure 4 According to at least one embodiment of the present disclosure, Figure 3 A schematic diagram of the correspondence between the lifetime of a storage object and the instructions that store the object in the case of .

[0014] Figure 5 A block diagram of an apparatus for allocating processor resources according to at least one embodiment of the present disclosure is shown.

[0015] Figure 6 A block diagram of an exemplary processor resource allocation apparatus according to at least one embodiment of the present disclosure is shown.

[0016] Figure 7 A schematic diagram of a non-transitory computer-readable storage medium according to at least one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0017] Reference will now be made in detail to specific embodiments of the present disclosure, examples of which are illustrated in the accompanying drawings. Although the present application will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit the present application to the described embodiments. On the contrary, it is intended to cover changes, modifications, and equivalents included within the spirit and scope of the present disclosure. It should be noted that the method steps described herein can all be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of the two.

[0018] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0019] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0020] In artificial intelligence scenarios, a large number of tensor operations are required to perform convolution, matrix multiplication and other operations to train and reason about artificial intelligence models, etc., for applications in natural language processing, image processing and computer vision, medical fields, recommendation systems and personalized push notifications, intelligent transportation and logistics, games and entertainment, etc.

[0021] However, in existing open source program compilers, which are usually designed for the central processing unit (CPU) architecture, the way they allocate registers or storage resources is not completely suitable for the artificial intelligence usage scenarios of tensor operations of general-purpose graphics processing units (GPGPUs) or graphics processing units (GPUs). Traditionally, storage resources are allocated only when storage objects that need to be allocated are encountered. At this time, only fragmented storage resources may be available for allocation, so it is difficult to allocate the entire tensor to the entire address of storage resources such as registers, which may in turn generate more fragments. Traditional compilers describe the data flow of programs in a relatively complex way, and the design of customized storage resource allocation algorithms based on this is difficult and not easy to optimize. Purely manual storage resource allocation methods place high demands on programmers and have low development efficiency.

[0022] The defects existing in the solutions in the above prior art are also the results obtained after careful study through practice and creative labor. The discovery process of the above problems and the solutions proposed for the above problems in at least one embodiment disclosed in the text below are all contributions to creativity.

[0023] At least one embodiment of the present disclosure proposes a processor resource allocation scheme in the program compilation process. The scheme can be used for a scenario where, for example, a GPGPU executes a program including tensor operations. During the program compilation process, the program's various storage objects' live intervals are analyzed by a structured control flow based on the program's instruction control flow, and the storage resources of each storage object are allocated and the storage resources are recovered for each instruction in the order from most to least (or from more to less) of the total storage amount of the storage objects in the survival state of the instruction. This scheme is simple in design and is more suitable for scenarios where a GPGPU executes a program including tensor operations. The scheme only needs to traverse the instruction control flow once to perform a lifetime analysis, reducing the complexity of the lifetime analysis caused by the unstructured control flow of the program. The storage object can also be allocated and recovered according to the lifetime of the storage object corresponding to the instruction, which can reduce the generation of storage resource fragments and improve the program execution efficiency and storage resource utilization effect. The scheme can uniformly perform centralized resource allocation of all storage resources, thereby improving allocation efficiency. At the same time, the resource allocation scheme can be applied to independent resource pools for different types of resources (needing to be recovered or not), and adapt to different allocation strategies. The resource allocation scheme also proposes two allocation schemes: the first allocation is for the non-temporary resources required for the lifetime of the cross-tensor instruction that performs operations on tensors (these resources are recycled after the lifetime ends); the second allocation is for the temporary resources required during the execution of the machine instructions generated by the lower() method inside the cross-tensor instruction (the temporary resources are recycled immediately after the execution of this tensor instruction), so as to adapt different allocation strategies to different types of resources and allocate storage resources reasonably and appropriately.

[0024] Figure 1 A flowchart of a method for allocating processor resources during a program compilation process according to at least one embodiment of the present disclosure is shown.

[0025] Here, the program may include, for example, a program with simplified control flow that uses tensors as processing objects, and may be written in programming languages ​​or expression systems designed for deep learning, high-performance computing, etc., such as the Triton language, the Tensor Virtual Machine (TVM), and the Tensor Expression (TensorExpression). The program usually includes multiple types of instructions, such as loop instructions, instructions of conditional branch structures, instructions of sequential execution structures, etc. These types of instructions may include specific instructions such as tensor matrix multiplication instructions, tensor convolution instructions, bias addition instructions, etc.

[0026] The program needs to be compiled before it is actually executed. The compilation process of the program is the process of converting the source code written in a high-level programming language (such as CUDA, etc.) into machine code that can be directly executed by the computer. This process is usually divided into several stages, including preprocessing, compilation, assembly, and linking. The preprocessing stage is the first stage of the compilation process, which mainly processes the precompilation instructions and macro definitions in the source code. The compilation stage converts the preprocessed source code into intermediate code or assembly code. The assembly stage converts the compiled assembly language code into machine language instructions (i.e., machine code). The linking stage links the target file (containing machine code) generated by the assembler with the standard library and other external code libraries to generate the final executable file. During the compilation process of the program, it is necessary to allocate physical addresses of storage resources for storage objects such as variables, arrays, matrices, tensors, etc. in the program to ensure that these storage objects have a certain location in the storage resources during the running of the program, so that the processor can efficiently access them and perform specific operations.

[0027] The execution subject of the method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, such as a computer device including a processing device and a storage device. In this embodiment, the processing device can be a CPU, a GPGPU or a GPU, and the storage device can be a random access memory (RAM), a read-only memory (ROM), a flash memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a floppy disk, a solid-state hard disk, a removable disk, a CD-ROM, a DVD-ROM, a Blu-ray disc, etc. The computer device is particularly suitable for allocating storage resources of a GPGPU or GPU processor that performs tensor operations during the compilation process of a tensor operation program. Of course, the computer device can also allocate storage resources to a CPU processor during the compilation process of a program running on the CPU.

[0028] Figure 1 The method for allocating processor resources during program compilation includes steps 110 and 120 .

[0029] In step 110, lifetimes of a plurality of storage objects in a plurality of instructions of a program to be compiled are obtained.

[0030] Storage objects can include variables, arrays, matrices, tensors, etc.

[0031] The lifetime of a first storage object among multiple storage objects includes the instruction sequence involved from the start instruction in which the first storage object is used for the first time to the end instruction in which the first storage object is used for the last time. The main purpose of lifetime acquisition is to determine the active interval of each storage object (such as a variable) in the program, that is, for example, in which instructions or basic blocks the variable is active (that is, used or may be used, but has not been redefined). Lifetime acquisition may include analyzing and obtaining the lifetimes of multiple storage objects in multiple instructions of the program during the program compilation process, or may include obtaining the lifetimes of multiple storage objects in multiple instructions of the program that have been analyzed from other places.

[0032] It should be noted that the lifetime in the present disclosure is not a time concept in a general sense, but a concept at the instruction level, that is, when a storage object is executed, it will be active. When these instructions are executed, the storage object will be used, and at this time, the storage object should have corresponding storage resources to store the data corresponding to the storage object. In the program compilation process (not the actual program execution process), storage resources need to be allocated for these storage objects in advance. Therefore, the program compilation process does not care about the specific time when the storage object is active when the program is running, but cares about which instruction or instructions some storage objects should be active or alive (i.e. defined or used) or inactive or dead (i.e. not used or redefined).

[0033] That is, in this article, the lifetime of a storage object includes multiple instructions, that is, in these multiple instructions, the storage object is active or alive. That is, in the lifetime (including instructions) of the first storage object, the first storage object is in a alive state. Here, it is noted that the "first" in the first storage object does not specifically refer to a specific storage object or the first storage object in any order, but generally refers to any storage object in multiple storage objects. That is, for each storage object in the multiple storage objects, it can have the same operation or definition as the first storage object.

[0034] In at least one embodiment, step 110 of obtaining lifetimes of a plurality of storage objects in a plurality of instructions of a program to be compiled includes the following steps.

[0035] Traverse the instruction control flow of the program to determine multiple preceding instructions and multiple following instructions of instructions that use a first storage object among multiple storage objects. Determine the preceding instruction of the start instruction of the first storage object being used for the first time among the multiple preceding instructions and the following instruction of the end instruction of the first storage object being used for the last time among the multiple following instructions. Determine the lifetime of the first storage object based on the preceding instruction of the start instruction of the first storage object being used for the first time and the following instruction of the end instruction of the first storage object being used for the last time. Obtain all instructions included during the lifetime of the first storage object as an instruction sequence corresponding to the first storage object in a survival state.

[0036] The instruction control flow of a program can represent an abstract data structure of the execution order and logical dependencies between instructions in the program. This can be done by statically analyzing the source code of the program, extracting the dependencies and execution order between instructions, and constructing the instruction control flow. The instruction control flow can be described by a structured control flow. The structured control flow can include the execution order (also called linearity) of the instructions in the program, the structure of branch selection and loop repetition. That is, the multiple instructions of the program can include one or more of the instructions of the sequential execution structure, the instructions of the conditional branch structure, and the instructions of the loop structure. In the sequential execution structure, the instructions are executed from top to bottom in the order in which they are written. If, else, else if and switch instructions and corresponding branch execution instructions are provided to implement the conditional branch structure. In the conditional branch structure, different branch instructions are executed according to the conditional judgment. For, while and do-while statements are provided to implement the loop structure. In the loop structure, some instructions are repeatedly executed until a specific termination condition is met. Because these three types of structures can be called recursively, and the irregular control flow (such as function call, exception handling, etc.) from any other basic instruction block to any other basic instruction block may not be described, this type of description method has good adaptability to artificial intelligence applications of tensor operations. In this way, the solution according to at least one embodiment of the present disclosure is more suitable for scenarios where GPGPU or GPU executes programs including tensor operations.

[0037] Note that control flow is often represented by a graph. A control flow graph usually includes nodes and edges. Nodes represent instructions or basic blocks of a program, and edges represent the transfer of control flow. For two nodes A and B in a control flow graph, if A dominates B, it means that every path from the program's entry node to B must pass through A. In this case, A is called the pre-dominant node of B, and the instruction represented by A is called the pre-dominant instruction of B. If A is the post-dominant node of B, it means that every path from B to the program exit must pass through A (or under some definitions, it is the path to a specific node). In this case, the instruction represented by A can be called the post-dominant instruction of B.

[0038] Figure 2A and Figure 2B A schematic diagram showing the instruction control flow of a program according to at least one embodiment of the present disclosure.

[0039] like Figure 2A As shown, the instruction control flow of the program includes the instruction beforeIf use a of the sequential execution structure, the instructions If, IfTrue use a, IfFalse use a of the conditional branch structure, and the instruction afterIf of the sequential execution structure. The instructions that use the storage object a (variable a) include the beforeIf use a instruction, the IfTrue use a instruction, and the IfFalseuse a instruction. The preceding controlling instruction of the beforeIf use a instruction is the previous instruction (not shown), and the succeeding controlling instruction is the If instruction. The preceding controlling instruction of the IfTrue use a instruction is If, and the succeeding controlling instruction is afterIf. The preceding controlling instruction of the IfFalse usea instruction is If, and the succeeding controlling instruction is afterIf. It can be seen that the common succeeding controlling instruction of the IfTrue use a instruction and the IfFalseuse a instruction is the afterIf instruction.

[0040] like Figure 2BAs shown, the instruction control flow of the program includes the instruction beforeloop usea of ​​the sequential execution structure, the instructions loop and loopBody use a of the loop repetition structure, and the instruction afterLoop of the sequential execution structure. The instructions that use the storage object a (variable a) include the beforeLoop use a instruction and the LoopBody use a instruction. The preceding controlling instruction of the beforeLoop use a instruction is the previous instruction (not shown), and the succeeding controlling instructions are the Loop instruction, the LoopBody use a instruction, and the afterLoop instruction. The preceding controlling instruction of the LoopBody use a instruction is Loop, and the succeeding controlling instruction is the afterLoop instruction. It can be seen that the common succeeding controlling instruction of the beforeLoop use a instruction and the LoopBody use a instruction is the afterLoop instruction.

[0041] Continue traversing to obtain all the preceding and following instructions of all the instructions that use the variable a, and some of these instructions may be common. In this way, the preceding and following instructions of the start instruction in which the storage object a is used for the first time among the multiple preceding and following instructions of the end instruction in which the storage object a is used for the last time among the multiple following instructions can be determined. Based on the preceding and following instructions of the start instruction in which the storage object a is used for the first time and the following instructions of the end instruction in which the storage object a is used for the last time, the lifetime of the storage object a can be determined, for example, all instructions between the preceding and following instructions of the start instruction in which the storage object a is used for the first time and the following instructions of the end instruction in which the storage object a is used for the last time (excluding the preceding and following instructions that do not use the storage object a), thereby obtaining all the instructions included during the lifetime of the first storage object as the instruction sequence corresponding to the first storage object in the survival state.

[0042] In this way, lifetime analysis can be performed by traversing the instruction control flow only once, reducing the complexity of lifetime analysis caused by the unstructured control flow of the program.

[0043] According to the lifetimes of the plurality of storage objects, a corresponding relationship between each instruction of the plurality of instructions and the storage objects in a live state may be determined.

[0044] Figure 3 A schematic diagram showing the correspondence between each instruction in a program segment and a storage object in a live state according to at least one embodiment of the present disclosure.

[0045] As described above, the lifetime of each storage object (the instructions in it) is obtained, and the correspondence between the storage object and the instruction can be obtained. However, in order to better implement the method of at least one embodiment of the present disclosure, reverse mapping can be performed, and the storage object corresponding to the instruction is conversely mapped to the instruction corresponding to the storage object.

[0046] like Figure 3 As shown, according to the lifetimes of the plurality of storage objects, the corresponding relationship between each instruction in the plurality of instructions and the storage objects in the survival state can be determined.

[0047] For example, the lifetime of storage object a includes instruction instr1(a) and instruction Loop() (including each instruction in Loop()).

[0048] The lifetime of storage object b includes instructions instr3(b,c), instr4(c), instr5(b), if, instr6(d) and instr7(b) in if instructions. Note that although instr4(c), instr5(b) and instr6(d) in if instructions do not actually use storage object b, storage object b cannot usually be released during the execution of these instructions, because the subsequent instr7(b) needs to use storage object b for the last time. Therefore, the lifetime of storage object b should include all instructions from the instruction where storage object b is used for the first time to the instruction where it is used for the last time. This makes it easier to design storage resource allocation plans and reserve enough storage resources to ensure the normal execution of the program.

[0049] The lifetime of the storage object c includes the instructions instr3(b,c) and instr4(c).

[0050] The lifetime of storage object d includes instruction Loop() (including each instruction in Loop()) and instr8(d). Note that although storage object d is only needed in instr6(d) in the first loop of instruction Loop(), the storage resources allocated for storage object d are needed thereafter and in subsequent loops. Therefore, the lifetime of storage object d is extended to instruction Loop() (including each instruction in Loop()), which makes it easier to design storage resource allocation schemes and reserve sufficient storage resources to ensure normal execution of the program.

[0051] So, like Figure 3 As shown, the storage object in the living state corresponding to each instruction can be obtained.

[0052] Figure 4 According to at least one embodiment of the present disclosure, Figure 3A schematic diagram of the correspondence between the lifetime of a storage object and the instructions that store the object in the case of .

[0053] like Figure 4 As shown, the lifetime of storage object a is shown as 410, the lifetime of storage object b is shown as 420, the lifetime of storage object c is shown as 430, and the lifetime of storage object d is shown as 440. The vertical dotted line passes through the storage objects in the survival state corresponding to the instructions.

[0054] Here, it is assumed that the data format of the storage objects a, b, and c is INT 16, and the capacity determined by the data format INT 16 is 2 bytes; the data format of the storage object d is INT 32, and the capacity determined by the data format INT 32 is 4 bytes.

[0055] The total storage capacity of the storage objects in the living state may be calculated by summing the capacity determined by the data format of each storage object in the living state.

[0056] Then, the total storage capacity of the storage objects in the survival state corresponding to each instruction can be determined. For example, the total storage capacity of the storage objects in the living state corresponding to the instruction instr1(a) is 2 bytes; the total storage capacity of the storage objects in the living state corresponding to the instruction Loop() is 6 bytes; the total storage capacity of the storage objects in the living state corresponding to the instruction instr2(a) is 6 bytes; the total storage capacity of the storage objects in the living state corresponding to the instruction instr3(b,c) is 10 bytes; the total storage capacity of the storage objects in the living state corresponding to the instruction instr4(c) is 10 bytes; the total storage capacity of the storage objects in the living state corresponding to the instruction instr5(b) is 8 bytes; the total storage capacity of the storage objects in the living state corresponding to the instruction if() is 8 bytes; the total storage capacity of the storage objects in the living state corresponding to the instruction instr6(d) is 6 bytes; the total storage capacity of the storage objects in the living state corresponding to the instruction instr7(b) is 8 bytes; the total storage capacity of the storage objects in the living state corresponding to the instruction instr8(d) is 4 bytes.

[0057] In this way, the relationship between the total storage capacity of storage objects in a survival state corresponding to multiple instructions can be determined, such as which instruction(s) corresponds to the largest or more total storage capacity of storage objects in a survival state (for example, at least greater than or equal to the first total storage capacity threshold).

[0058] The fact that the total storage capacity of the storage objects in the survival state corresponding to the instruction is the largest or more means that when the instruction is executed, storage resources need to be allocated to the storage objects with the largest or more total storage capacity. If the program execution can be prioritized (for example, the earlier in time or the higher in priority), sufficient storage resources can be allocated to the storage objects with the largest or more total storage capacity. At this time, the storage resources are not allocated too much, and the whole block of storage resources can be allocated as much as possible, instead of only fragmented storage resources available for allocation. However, if the allocation is started only when the storage objects that need to be allocated storage resources are encountered, or resources are allocated to all storage objects at the same priority, as in the traditional way, it will lead to the running of subsequent instructions, especially the instruction that needs to allocate storage resources for the storage objects with the largest or more total storage capacity, there may not be enough storage resources, or there is no whole block of storage resources, only fragmented storage resources are available for allocation, which may cause the storage resources to be more fragmented, reducing the program execution efficiency and the storage resource utilization effect. According to at least one embodiment of the present disclosure, the generation of storage resource fragments can be reduced, and the program execution efficiency and storage resource utilization effect can be improved.

[0059] return Figure 1 In step 120, storage resources are allocated to storage objects in a plurality of instructions that are in a survival state.

[0060] In some embodiments, the total storage capacity of the storage objects in a survival state corresponding to each instruction in the multiple instructions is determined based on the corresponding relationship. The total storage capacity of the storage objects in a survival state of the first instruction in the multiple instructions may be more than the storage resources allocated to the storage objects in a survival state in the second instruction in the multiple instructions. The allocation of storage resources to the storage objects in a survival state in the first instruction in the multiple instructions takes precedence over the allocation of storage resources to the storage objects in a survival state in the second instruction in the multiple instructions.

[0061] like Figure 3 and Figure 4As shown, the total storage capacity of the storage objects in the live state in instr3(b,c) and instr4(c) is 10 bytes. The total storage capacity of the storage objects in the live state in Instr5(b) and the if instruction is 8 bytes. The total storage capacity of the storage objects in the live state in instr3(b,c) and instr4(c) is greater than the storage objects in the live state in Instr5(b) and the if instruction. In this way, the allocation of storage resources to the live storage objects a, b, c, d in instr3(b,c) and instr4(c) takes precedence over the allocation of storage resources to the live storage objects in Instr5(b) and the if instruction. For example, in terms of time, the storage resources are allocated starting from the live storage objects a, b, c, d in instr3(b,c) and instr4(c).

[0062] In some embodiments, in the first instruction, the total storage capacity of the storage objects in the survival state is greater than or equal to a first total storage capacity threshold, and in the second instruction, the total storage capacity of the storage objects in the survival state is less than the first total storage capacity threshold.

[0063] In this way, the priority of allocating storage resources to the storage object in the first instruction in the survival state may be higher than the priority of allocating storage resources to the storage object in the second instruction in the survival state. That is, allocating storage resources to the storage object in the survival state in the first instruction among the multiple instructions takes precedence over allocating storage resources to the storage object in the survival state in the second instruction among the multiple instructions.

[0064] Note that the first instruction or the second instruction here may also refer to any one or more instructions without any order or size restrictions, as long as the conditions are met.

[0065] Here, the first total storage threshold can be preset. In addition, the case where the total storage of the storage objects in the survival state in the first instruction is the largest can also be included in the case where the total storage of the storage objects in the survival state in the first instruction is greater than or equal to the first total storage threshold, because the first total storage threshold can be a predetermined value.

[0066] like Figure 3 and Figure 4 As shown in the figure, the total storage capacity of the storage objects in the live state in instr3(b,c) and instr4(c) is 10 bytes. The total storage capacity of the storage objects in the live state in instr5(b) and the if instruction is 8 bytes.

[0067] For example, if the first total storage capacity threshold is 9 bytes, the priority of allocating storage resources to the storage objects in instr3(b,c) and instr4(c) whose total storage capacity is greater than or equal to 9 is higher than the priority of allocating storage resources to the storage objects in other instructions (such as Instr5(b) and if instructions and other instructions) whose total storage capacity is less than 9. For example, in terms of time, storage resources are allocated starting from the storage objects a, b, c, and d in instr3(b,c) and instr4(c) that are in a living state.

[0068] In some embodiments, when the total storage capacity of the storage objects in the surviving state in the first instruction is the largest, the storage resources of the storage objects in the surviving state in the first instruction are allocated preferentially.

[0069] like Figure 3 and Figure 4 As shown, it can be considered that the total storage capacity of the storage objects in the survival state in instr3(b,c) and instr4(c) is the largest (10 bytes).

[0070] In this way, storage resources can be allocated to storage objects a, b, c, d in instruction instr3(b, c) or instruction instr4(c) first. For example, storage resources are allocated first from storage objects a, b, c, d in instr3(b, c) and instr4(c) that are in a live state.

[0071] In one embodiment, the allocation of storage resources to the storage objects in the survival state in multiple instructions is performed by allocating a whole block of storage addresses to the storage objects. The storage objects that are allocated with higher priority (or earlier in time) will be allocated with as many whole blocks of storage addresses as possible. For example, the register physical address [0, 1023] is allocated to the storage object a, the register physical address [1024, 2047] is allocated to the storage object b, the register physical address [2048, 4095] is allocated to the storage object c, and the register physical address [4095, 8191] is allocated to the storage object d. In this way, a whole block (i.e., continuous) of register addresses can be allocated to each storage object (i.e., the register addresses allocated to each storage object are continuous), and a whole block of register addresses can also be allocated to all storage objects (i.e., the register addresses allocated to all storage objects are also continuous), which can further reduce the generation of fragments.

[0072] After allocating storage resources to storage objects a, b, c, and d of instruction instr3(b,c) or instruction instr4(c), the storage objects to which storage resources have been allocated are spread to all instructions corresponding to the storage objects (i.e., the storage objects of each instruction should be able to use the allocated storage resources to store data), ensuring that the data of the storage object can be stored using the allocated storage resources when the instruction that uses the storage object for the first time is run. Figure 3 As shown, after allocating storage resources to the storage objects a, b, c, d of instruction instr3(b,c) or instruction instr4(c), Figure 3 All storage objects in the instructions shown have been allocated storage resources. Subsequent storage resource allocation can be done on unallocated storage objects (e.g. Figure 3 The storage object in other instructions not shown) is performed.

[0073] Notice, Figure 3 and Figure 4 Only storage objects a, b, c, and d are shown in the figure, but other storage objects in other instructions may also exist in the entire program. The allocation priority can be determined based on the lifetime of all storage objects in the entire program, so as to allocate and spread storage resources.

[0074] In the process of allocating storage resources of storage objects other than the storage object in the first instruction, there may be a variety of allocation orders. Several examples of allocation orders are given below, but they are not limiting.

[0075] For example, after the storage object in the first instruction is preferentially allocated storage resources, the instructions before the first instruction (for example Figure 3 The storage objects in the program instructions before all the program instructions shown in the figure are allocated storage resources in the order of instructions from the back to the front; according to the instruction control flow of the program, the instructions after the first instruction (for example Figure 3 After resources are allocated to a storage object, they are also diffused to all instructions having the storage object in the above diffusion manner.

[0076] For example, the instructions other than the first instruction in the plurality of instructions may be allocated storage resources for other storage objects other than the allocated storage objects in descending order of the total storage amount of the storage objects. Figure 3 The total amount of storage objects that need to be allocated in the program instructions shown is 10 bytes, which is the maximum. Figure 3After the storage resources of the storage objects a, b, c, d that need to be allocated in the program instructions shown are allocated, the storage resources allocated to the storage objects a, b, c, d are diffused to a, b, c, d in other program instructions, and then the total storage capacity of the storage objects (e.g., e, f) that need to be allocated in a certain program instruction not shown is 8 bytes, and the storage objects (e.g., g, h) that need to be allocated in other program instructions are 6 bytes... Then the storage objects in a certain program instruction not shown with a total storage capacity of 8 bytes are allocated storage resources, and then the storage objects in other program instructions not shown with a total storage capacity of 6 bytes are allocated storage resources, and so on. Note that after resources are allocated to a storage object, they are also diffused to all instructions with the storage object in the above diffusion method, and the next allocation does not need to allocate resources to the storage objects that have already been allocated storage resources. How to allocate a whole block of storage addresses to as many storage objects as possible to reduce the generation of storage resource fragments.

[0077] The resource allocation plan also proposes two allocation plans.

[0078] The first allocation of storage resources required for the lifetime of cross-tensor instructions that operate on tensors. Figure 1 , Figure 2A , Figure 2B , Figure 3-Figure 4 In the example shown, multiple instructions (such as instr1(a)) are operations on tensors. Figure 3 The storage objects a, b, c, and d in are tensors. Therefore, the first allocation is the storage resources required for the lifetime of cross-tensor instructions (instr1(a), etc.). Here, once the required storage resources are allocated for the lifetime of cross-tensor instructions (instr1(a), etc.), the storage resources have been allocated for all cross-tensor instructions before the program starts executing. Before the last execution of a storage object in the entire program is completed, the storage resource will not be reclaimed until the last execution of the storage object in the entire program is completed (that is, the lifetime of the storage object ends), and the storage resource is reclaimed.

[0079] The second allocation is to allocate temporary resources required during the execution of the machine instructions generated by the lower() method of the cross-tensor instruction (the temporary resources here refer to those that are immediately recycled after the execution of the tensor instruction). Lower() is a stage in the back end of the compiler, which can be responsible for converting the intermediate representation (IR) obtained from the program code into a low-level representation (for example, binary code) that is closer to the target machine instruction. The Lower() stage may involve tasks such as register allocation, instruction selection, and instruction scheduling. That is, for the machine instruction generated by the third instruction among multiple instructions, temporary storage resources can also be allocated to the storage object in the machine instruction, wherein the temporary storage resources are recycled after the third instruction is executed. Here, the third instruction can refer to any instruction as long as it can generate a machine instruction by, for example, the lower() method of the lower function. Usually any instruction in a program can be further decomposed into machine instructions.

[0080] For example, the tensor matrix multiplication instruction Tensor(a)*Tensor(b) instruction can generate machine instructions by lowering the function lower() method, which can include a multiplication instruction between an element (or first element) a1 in tensor a and an element (or second element) b1 in tensor b, and at this time, temporary resources are allocated to the storage resources required by the two storage objects of the first element a1 and the second element b1, so as to temporarily store the values ​​of the first element a1 and the second element b1, and after the multiplication instruction between the first element a1 and the second element b1 is executed, the temporary resources can be immediately recycled. The machine instruction can also include a multiplication instruction between another element (or third element) a2 in tensor a and another element (or fourth element) b2 in tensor b, and at this time, temporary resources are allocated to the storage resources required by the two storage objects of the third element a2 and the fourth element b2, so as to temporarily store the values ​​of the third element a2 and the fourth element b2, and after the multiplication instruction between the first element a1 and the second element b1 is executed, the temporary resources can be immediately recycled. And so on, which will not be described one by one here.

[0081] In other words, in the process of allocating storage resources, at least one of a temporary storage resource that needs to be reclaimed immediately after an instruction is executed and a storage resource that is reclaimed after the lifetime ends can be allocated to each of the multiple storage objects according to the lifetime of the multiple storage objects. As described above, there are many different types of storage resources, and a resource pool can be set for each type of storage resource, so that resources are allocated in the corresponding resource pool.

[0082] In this way, different allocation strategies can be adapted to different types of resources that need to be recycled and those that do not need to be recycled, so that storage resources can be allocated reasonably and appropriately.

[0083] In summary, this solution can uniformly perform centralized resource allocation of all storage resources, thereby improving allocation efficiency.

[0084] Although the example of the solution according to at least one embodiment of the present disclosure is for allocating storage resources in a GPGPU or GPU, the solution can also be applied to allocating storage resources in a CPU, and therefore does not limit the storage resources of a processing unit.

[0085] Figure 5 A block diagram of an apparatus 500 for allocating processor resources during a program compilation process according to at least one embodiment of the present disclosure is shown.

[0086] like Figure 5 As shown, the apparatus 500 for allocating processor resources during program compilation includes an acquisition module 510 and an allocation module 520 .

[0087] The acquisition module 510 can be configured to acquire the lifetimes of multiple storage objects among multiple instructions of a program to be compiled, wherein the lifetime of a first storage object among the multiple storage objects includes a sequence of instructions involved from a start instruction when the first storage object is used for the first time to an end instruction when the first storage object is used for the last time, wherein, during the lifetime of the first storage object, the first storage object is in a survival state.

[0088] The allocation module 520 can be configured to allocate storage resources to storage objects in a survival state in multiple instructions, wherein allocating storage resources to storage objects in a survival state in a first instruction among the multiple instructions takes precedence over allocating storage resources to storage objects in a survival state in a second instruction among the multiple instructions, wherein the total storage capacity of the storage objects in a survival state in the first instruction is greater than the total storage capacity of the storage objects in a survival state in the second instruction.

[0089] In one embodiment, in the first instruction, the total storage capacity of the storage objects in the survival state is greater than or equal to a first total storage capacity threshold, and in the second instruction, the total storage capacity of the storage objects in the survival state is less than the first total storage capacity threshold.

[0090] In one embodiment, the acquisition module 510 can be configured to traverse the instruction control flow of the program, determine multiple preceding instructions and multiple following instructions of instructions that use a first storage object among multiple storage objects; determine the preceding instruction of the start instruction where the first storage object is used for the first time among the multiple preceding instructions and the following instruction of the end instruction where the first storage object is used for the last time among the multiple following instructions; determine the lifetime of the first storage object based on the preceding instruction of the start instruction where the first storage object is used for the first time and the following instruction of the end instruction where the first storage object is used for the last time; and treat all instructions included during the lifetime of the first storage object as an instruction sequence corresponding to the first storage object in a survival state.

[0091] In one embodiment, the allocation module 520 can be configured to: determine the correspondence between each instruction of the multiple instructions and the storage objects in a survival state according to the lifetime of the multiple storage objects; and determine the total storage capacity of the storage objects in a survival state corresponding to each instruction of the multiple instructions according to the correspondence.

[0092] In one embodiment, the total storage amount of the storage objects in the survival state in the first instruction is the largest.

[0093] In one embodiment, the allocation module 520 can be configured to: allocate storage resources to the storage objects in the first instruction; allocate storage resources to the storage objects in the instructions before the first instruction in a sequence from the back to the front according to the instruction control flow of the program; allocate storage resources to the storage objects in the instructions after the first instruction in a sequence from the front to the back according to the instruction control flow of the program.

[0094] In one embodiment, the allocation module 520 can be configured to: allocate storage resources to the storage objects in the first instruction; and allocate storage resources to other storage objects except the allocated storage objects in the multiple instructions except the first instruction in order of the total storage capacity of the storage objects from most to least.

[0095] In one embodiment, an operand of one or more of the plurality of instructions may be a tensor.

[0096] In one embodiment, the plurality of instructions may include one or more of instructions of a sequential execution structure, instructions of a conditional branch structure, and instructions of a loop structure.

[0097] In one embodiment, the allocation module 520 may also be configured to: allocate temporary storage resources to storage objects in a machine instruction generated by a third instruction among the multiple instructions, wherein the temporary storage resources are reclaimed after the third instruction is executed.

[0098] In one embodiment, the allocation module 520 can be configured to allocate, according to the lifetime of multiple storage objects, to each of the multiple storage objects at least one of temporary storage resources that need to be reclaimed immediately after an instruction is executed during the execution of the program and storage resources that need to be reclaimed only after the entire program is executed.

[0099] In one embodiment, the allocation module 520 may be configured to allocate storage resources to storage objects in a live state in multiple instructions by allocating a whole block of storage addresses to the storage objects.

[0100] Thus, according to at least one embodiment of the present disclosure, for example, for a scenario where a GPGPU executes a program including tensor operations, during the program compilation process, the lifetime of each storage object of the program can be analyzed by a structured control flow based on the instruction control flow of the program, and the storage resources of its respective storage objects are allocated and the storage resources are recovered for each instruction in the order from most to least (or from more to less) of the total storage amount of the storage objects in the survival state of the instruction. This solution is simple in design and is more suitable for the scenario where the GPGPU executes a program including tensor operations. The solution only needs to traverse the instruction control flow once to perform lifetime analysis, reducing the complexity of lifetime analysis caused by the unstructured control flow of the program. The storage objects can also be allocated and recovered according to the lifetime of the storage objects corresponding to the instructions, which can reduce the generation of storage resource fragments and improve the program execution efficiency and storage resource utilization effect. The solution can uniformly perform centralized resource allocation of all storage resources, thereby improving allocation efficiency. At the same time, the resource allocation scheme can be applied to independent resource pools for different types of resources (needing to be recovered or not), and adapt to different allocation strategies. This resource allocation scheme also proposes two allocation schemes: the first allocation is for non-temporary resources required for the lifetime of cross-tensor instructions that perform operations on tensors; the second allocation is for temporary resources required during the execution of machine instructions generated by the cross-tensor instructions through the function reduction method (the temporary resources are recycled immediately after the tensor instruction is executed), so as to adapt different allocation strategies to different types of resources and allocate storage resources reasonably and appropriately.

[0101] Figure 6 A block diagram of an apparatus 600 for allocating processor resources during an exemplary program compilation process according to at least one embodiment of the present disclosure is shown.

[0102] The device 600 for allocating processor resources during program compilation may include a processing device 610 and a storage device 620, wherein the storage device 620 is coupled to the processing device 610 and stores computer instructions therein for performing the steps of each method of at least one embodiment of the present disclosure when executed by the processing device 610.

[0103] The processing device 610 may include, but is not limited to, one or more CPUs or GPGPUs or GPUs or microprocessing units, etc., but is typically a CPU.

[0104] The storage device 620 may include, but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disk, etc.).

[0105] In addition, if necessary, the apparatus for allocating processor resources during the program compilation process may also include (but not limited to) other components, such as a bus, a display, and input / output devices (eg, keyboard, mouse, speaker, etc.).

[0106] In one embodiment, the at least one computer instruction may also be compiled into or constitute a computer program product or software product, wherein one or more computer instructions, when executed by a processing device, perform the various functions and / or method steps in the embodiments described in the present technology.

[0107] Figure 7 A schematic diagram of a non-transitory computer-readable storage medium according to at least one embodiment of the present disclosure is shown.

[0108] like Figure 7 As shown, the non-transitory computer-readable storage medium 720 stores instructions, such as computer instructions 710. When the computer instructions 710 are executed by the processing device, the various methods described above can be executed. The non-transitory computer-readable storage medium includes, but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, register, computer storage medium (such as hard disk, floppy disk, solid state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.), etc. For example, the non-transitory computer-readable storage medium 720 can be connected to a computing device such as a computer, and then, when the computing device executes the computer instructions 710 stored on the computer-readable storage medium 720, the various methods described above can be performed.

[0109] The present disclosure may also include electronic devices, including Figure 5 and 6 A device for allocating processor resources during program compilation is described.

[0110] The present disclosure may also include a computer program product, wherein the computer program product can perform the methods, steps, and operations presented herein. For example, such a computer program product can be a computer software package, computer code instructions, a computer-readable tangible medium having computer instructions tangibly stored (and / or encoded) thereon, which instructions can be executed by a processing device to perform the operations described herein. The computer program product can include packaging materials.

[0111] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably therewith. The words "such as / for example" used herein refer to the phrase "such as / for example but not limited to", and can be used interchangeably therewith.

[0112] The step flow charts and the above method descriptions in this disclosure are only illustrative examples and are not intended to require or imply that the steps of each embodiment must be performed in the order given. As will be appreciated by those skilled in the art, the order of the steps in the above embodiments can be performed in any order. Words such as "thereafter", "then", "next", etc. are not intended to limit the order of steps; these words are only used to guide the reader through the description of these methods. In addition, any reference to a singular element, such as using the article "one", "one", or "the" is not to be construed as limiting the element to the singular.

[0113] In addition, the steps and devices in the various embodiments of this document are not limited to being implemented in a certain embodiment. In fact, according to the concept of this disclosure, relevant partial steps and partial devices in the various embodiments of this document can be combined to conceive new embodiments, and these new embodiments are also included in the scope of this disclosure.

[0114] The above methods may be implemented in hardware, software, firmware or any combination thereof.

[0115] In addition, modules and / or other appropriate means for performing the methods and techniques described herein can be downloaded from a server via wireless communication when appropriate. Alternatively, the various methods described herein can be provided via a storage component so that the various methods are available when coupled to the storage component. In addition, any other appropriate technology for providing the methods and techniques described herein to a device can be utilized.

[0116] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit at least one embodiment of the present disclosure to the form disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for allocating processor resources, comprising: Acquire lifetimes of multiple storage objects in multiple instructions of a program to be compiled, wherein the lifetime of a first storage object in the multiple storage objects includes an instruction sequence involved from a start instruction in which the first storage object is used for the first time to an end instruction in which the first storage object is used for the last time, and during the lifetime of the first storage object, the first storage object is in a live state; Allocate storage resources to storage objects in a survival state among the multiple instructions, wherein allocating storage resources to storage objects in a survival state in a first instruction among the multiple instructions takes precedence over allocating storage resources to storage objects in a survival state in a second instruction among the multiple instructions, wherein the total storage capacity of storage objects in a survival state in the first instruction is greater than the total storage capacity of storage objects in a survival state in the second instruction.

2. The method according to claim 1, wherein: In the first instruction, the total storage capacity of the storage objects in the survival state is greater than or equal to a first total storage capacity threshold, and in the second instruction, the total storage capacity of the storage objects in the survival state is less than the first total storage capacity threshold.

3. The method according to claim 1, wherein: The total storage capacity of the storage objects in the living state is calculated by the sum of the capacities determined by the data format of each storage object in the living state.

4. The method according to any one of claims 1 to 3, wherein: The allocating storage resources to the storage objects in the plurality of instructions that are in a live state comprises: Determining, according to the lifetimes of the plurality of storage objects, a correspondence between each instruction in the plurality of instructions and a storage object in a survival state; The total storage capacity of the storage objects in a survival state corresponding to each instruction in the multiple instructions is determined according to the corresponding relationship.

5. The method according to any one of claims 1 to 3, wherein: The total storage capacity of the storage objects in the survival state in the first instruction is the largest.

6. The method according to any one of claims 1 to 3, wherein: The allocating storage resources to the storage objects in the plurality of instructions that are in a live state comprises: Allocating storage resources to the storage object in the first instruction; According to the instruction control flow of the program, storage resources are allocated to the storage objects in the instructions before the first instruction in the order of the instructions from the back to the front; According to the instruction control flow of the program, storage resources are allocated to storage objects in the instructions after the first instruction in sequence from the front to the back of the instructions.

7. The method according to any one of claims 1 to 3, wherein: The allocating storage resources to the storage objects in the plurality of instructions that are in a live state comprises: Allocating storage resources to the storage object in the first instruction; For instructions other than the first instruction among the multiple instructions, storage resources of other storage objects other than the allocated storage objects are allocated in descending order of the total storage capacity of the storage objects.

8. The method according to any one of claims 1 to 3, wherein: An operand of one or more of the plurality of instructions is a tensor.

9. The method according to any one of claims 1 to 3, wherein: The multiple instructions include one or more of instructions of a sequential execution structure, instructions of a conditional branch structure, and instructions of a loop structure.

10. The method according to any one of claims 1 to 3, further comprising: For a machine instruction generated by a third instruction among the multiple instructions, a temporary storage resource is allocated to a storage object in the machine instruction, wherein the temporary storage resource is reclaimed after the third instruction is executed.

11. The method according to any one of claims 1 to 3, wherein: The allocating of storage resources to the storage objects in the multiple instructions that are in a live state is performed by allocating a whole block of storage addresses to the storage objects.

12. A device for allocating processor resources, comprising: an acquisition module, configured to acquire lifetimes of a plurality of storage objects among a plurality of instructions of a program to be compiled, wherein the lifetime of a first storage object among the plurality of storage objects includes an instruction sequence involved from a start instruction in which the first storage object is used for the first time to an end instruction in which the first storage object is used for the last time, and during the lifetime of the first storage object, the first storage object is in a survival state; An allocation module is configured to allocate storage resources to storage objects in a survival state in the multiple instructions, wherein allocating storage resources to storage objects in a survival state in a first instruction among the multiple instructions takes precedence over allocating storage resources to storage objects in a survival state in a second instruction among the multiple instructions, wherein the total storage capacity of storage objects in a survival state in the first instruction is greater than the total storage capacity of storage objects in a survival state in the second instruction.

13. A device for allocating processor resources, comprising: a storage device storing computer instructions; At least one processing device configured to execute the computer instructions in the storage device to perform the method according to any one of claims 1-11.

14. An electronic device comprising the apparatus for allocating processor resources according to claim 12 or 13.

15. A non-transitory computer readable storage medium having computer instructions stored thereon, in, When the computer instructions are executed by a processing device, the processing device is caused to perform the method according to any one of claims 1-11.

Images