A register overflow optimization method, device, storage medium and program product

Abstract

The embodiments of the present application provide a method, device, storage medium, and program product for optimizing register overflow, relating to the field of chip technology. The method includes: during the operator compilation process, performing register allocation for the target thread that executes the operator. If register overflow occurs during the register allocation process, write the overflowed virtual register to the target stack space corresponding to the target thread in the on-chip cache. The present application sets the corresponding target stack space for the target thread in the on-chip cache. Compared with the access speed of the video memory, the access speed of the on-chip cache is faster. Therefore, when register overflow occurs in the register allocation for the target thread, write the overflowed virtual register to the target stack space corresponding to the target thread in the on-chip cache, so that the access speed to the stack space is faster, thereby reducing the latency caused by register overflow and further improving the operator performance.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of chip technology, and in particular, to a method, device, storage medium, and program product for optimizing register overflow. Background Art

[0002] An artificial intelligence model generally refers to a neural network model trained for inference prediction, such as an image inference model, a speech inference model, etc. The operations of an artificial intelligence model can be implemented by operators in a computation graph. An operator refers to various operations performed on tensors of each layer in an artificial intelligence model. For example, the convolution operation performed by the convolution layer of an artificial intelligence model on the input data of the artificial intelligence model is a convolution operator.

[0003] When performing register allocation for an operator, the phenomenon of register overflow often occurs. When register overflow occurs, the overflowing virtual registers are written out to the video memory. Since the video memory is a lower-level memory in an artificial intelligence chip and the access speed of this memory is relatively slow, the speed of writing the overflowing virtual registers to the video memory is relatively slow, resulting in a decrease in the performance of the operator. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, storage medium, and program product for optimizing register overflow, which are used to reduce the latency caused by register overflow and thereby improve the performance of the operator.

[0005] On the one hand, the embodiments of the present application provide a method for optimizing register overflow, including:

[0006] During the operator compilation process, perform register allocation for the target thread that executes the operator;

[0007] If register overflow occurs during the process of performing the register allocation, write the overflowing virtual registers to the target stack space corresponding to the target thread in the on-chip cache.

[0008] On the one hand, the embodiments of the present application provide a device for optimizing register overflow, including:

[0009] A register allocation module, configured to perform register allocation for the target thread that executes the operator during the operator compilation process;

[0010] A writing module, configured to write the overflowing virtual registers to the target stack space corresponding to the target thread in the on-chip cache if register overflow occurs during the process of performing the register allocation.

[0011] Optionally, it further includes a reading module;

[0012] The reading module is specifically configured to:

[0013] During the execution of the operator, read the overflowed virtual register from the target stack space.

[0014] Optionally, among the multiple physical registers associated with the target thread, there is a stack pointer register for pointing to the target stack space;

[0015] The writing module is specifically configured to:

[0016] If a register overflow occurs during the execution of the register allocation, write the overflowed virtual register to the target stack space pointed to by the stack pointer register.

[0017] Optionally, among the multiple physical registers associated with the target thread, there is a stack pointer register for pointing to the starting address of the target stack space;

[0018] The writing module is specifically configured to:

[0019] If a register overflow occurs during the execution of the register allocation, write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register.

[0020] Optionally, the writing module is specifically configured to:

[0021] If a register overflow occurs during the execution of the register allocation and the register overflow occurs for the first time, obtain the starting address of the target stack space and save the starting address of the target stack space in the stack pointer register;

[0022] Write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register.

[0023] Optionally, the target thread is any one thread in the thread group, and each thread in the thread group corresponds to a stack space in the on-chip cache;

[0024] The writing module is specifically configured to:

[0025] Based on the initial address of the on-chip cache, the number of the target thread in the thread group, and the stack space size allocated for each thread, determine the starting address of the target stack space.

[0026] Optionally, the writing module is specifically configured to:

[0027] If a register overflow occurs during the execution of the register allocation process, and the storage space of the on-chip cache meets the register overflow requirements, the overflowed virtual register is written out to the starting address of the target stack space pointed to by the stack pointer register.

[0028] Optionally, the writing module is further configured to:

[0029] If a register overflow occurs during the execution of the register allocation process, and the storage space of the on-chip cache does not meet the register overflow requirements, the overflowed virtual register is written out to the video memory.

[0030] On the one hand, an embodiment of the present application provides a computer device, including a memory, an artificial intelligence chip, and a computer program stored on the memory and executable on the artificial intelligence chip. When the artificial intelligence chip executes the computer program, the steps of the above-mentioned register overflow optimization method are implemented.

[0031] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program executable by a computer device. When the computer program runs on the computer device, the computer device is enabled to execute the steps of the above-mentioned register overflow optimization method.

[0032] On the one hand, an embodiment of the present application provides a computer program product. The computer program product includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer device, the computer device is enabled to execute the steps of the above-mentioned data access method.

[0033] In the embodiment of the present application, a corresponding target stack space is set for the target thread in the on-chip cache. Compared with the access speed of the video memory, the access speed of the on-chip cache is faster. Therefore, when a register overflow occurs in the register allocation for the target thread, the overflowed virtual register is written out to the target stack space corresponding to the target thread in the on-chip cache, so that the access speed to the stack space is faster, thereby reducing the latency caused by the register overflow and further improving the operator performance. Description of the Drawings

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0035] Figure 1 It is a schematic structural diagram of an artificial intelligence chip provided by an embodiment of the present application;

[0036] Figure 2 Flow schematic of a register overflow optimization method provided by an embodiment of the present application Figure 1 ;

[0037] Figure 3 Schematic diagram of a stack pointer register provided by an embodiment of the present application;

[0038] Figure 4 Flow schematic of a register overflow optimization method provided by an embodiment of the present application Figure 2 ;

[0039] Figure 5 Schematic diagram of the structure of a register overflow optimization device provided by an embodiment of the present application;

[0040] Figure 6 Schematic diagram of the structure of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0041] In order to make the objectives, technical solutions and beneficial effects of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] Refer to Figure 1 , which is a structural diagram of an artificial intelligence chip applicable to an embodiment of the present application. The artificial intelligence chip 100 at least includes: a video memory 101 and a plurality of programmable multi-processors 102. Among them, the programmable multi-processor 102 may be a Streaming Processing Cluster (SPC for short).

[0043] Each programmable multi-processor 102 includes: a plurality of computing units 104. Each computing unit 104 includes: an on-chip cache 103 and a plurality of execution units 105. The plurality of execution units 105 inside the computing unit 104 share the on-chip cache 103.

[0044] Each execution unit 105 includes: physical registers 106 of multiple threads in a thread group. Multiple threads can execute concurrently, and each thread corresponds to one or more physical registers 106.

[0045] The video memory 101 can be a High Bandwidth Memory (HBM) or the like, or can be other types of memories. The on-chip cache 103 is a temporary memory, which has a smaller capacity than the video memory 101 but a faster data exchange speed than the video memory 101. The on-chip cache 103 can be a Group-Shared Memory (GSM), a Gemm Main Buffer (GMB), etc.

[0046] In addition to the above structures, the artificial intelligence chip 100 in this application may further include other structures, and specific limitations are not made in this application.

[0047] The artificial intelligence chip 100 can be a Graphics Processing Unit (GPU), a General-purpose computing on graphics processing units (GPGPU), a Domain Specific Architecture (DSA), etc.

[0048] In practical applications, register allocation is a fundamental problem in compiler technology and plays a crucial role in the performance of the compiled code. When performing register allocation for an operator, the phenomenon of register spill often occurs. When register spill occurs, the spilled virtual registers are written out to the video memory. Since the video memory is a lower-level memory in the artificial intelligence chip and the access speed of these memories is relatively slow, the speed of writing the spilled virtual registers to the video memory is relatively slow. Thus, even if the number of register spills is small, the additional overhead of accessing the video memory caused by register spill will result in a decline in program performance, thereby leading to a decline in operator performance.

[0049] In view of this, based on Figure 1 the architecture diagram of the artificial intelligence chip shown, a flow of a register spill optimization method is provided, as shown in Figure 2 shown, and the flow of this method is executed by the artificial intelligence chip, including the following steps:

[0050] Step 201, during the operator compilation process, perform register allocation for the target thread that executes the operator.

[0051] Specifically, in the artificial intelligence chip, a preset number of physical registers are pre-allocated for multiple threads respectively, and the number of physical registers allocated to each thread is associated with the specific hardware attributes of the artificial intelligence chip.

[0052] In this application, the target thread can be any one of the multiple threads described above. Therefore, the artificial intelligence chip includes a preset number of physical registers pre-allocated to the target thread.

[0053] During the operator compilation process, the kernel function of the operator is converted from the intermediate representation in the form of a syntax tree to the intermediate representation in the three-address form. For the intermediate representation in the form of a syntax tree, the dependency relationships between the operands and results of operations can all be expressed through a tree structure. After converting to the intermediate representation in the three-address form, a large number of temporary variables need to be created to express the dependency relationships between the operands and results of operations. Then, corresponding virtual registers are allocated for all variables (including temporary variables and variables in the original kernel function), that is, virtual registers are used to represent variables.

[0054] Next, register allocation is performed on the target thread that executes the operator, that is, each of the above-generated virtual registers is mapped to the physical registers corresponding to the target thread (i.e., the preset number of physical registers pre-allocated to the target thread), which means writing the variables associated with each virtual register to the physical registers mapped by the virtual registers.

[0055] In this way, during the operator execution process, when a variable associated with a certain virtual register is needed, the variable is read from the physical register mapped by the virtual register and then executed.

[0056] Since the number of virtual registers is infinite, but the number of physical registers is finite. Therefore, during the register allocation process, if the number of physical registers is insufficient, a register overflow phenomenon will occur.

[0057] For example, assume that the target thread corresponds to 128 physical registers, and the number of virtual registers associated with the operator is 130. After allocating a physical register for each virtual register, there are still 2 virtual registers without physical register allocation, and at this time, a register overflow phenomenon occurs.

[0058] In addition, in this application, the target thread can be any one of the threads in the thread group. Multiple threads in the thread group execute the operator concurrently, and each thread in the thread group is applicable to the register overflow optimization method in this application. Of course, the target thread can also not belong to any thread group, and this application does not make specific limitations in this regard.

[0059] Step 202, if a register overflow occurs during the register allocation process, write the overflowed virtual registers to the target stack space corresponding to the target thread in the on-chip cache.

[0060] Specifically, the overflowed virtual registers are the virtual registers in the operator for which no physical registers are allocated. The stack (also known as the stack or overflow stack) is the storage medium for register overflow. Using the stack as the storage medium for register overflow has the advantage of strong scalability. In this application, storage space is pre-allocated for the stack in the on-chip cache, and this storage space is the stack space.

[0061] For a thread group, each thread in the thread group corresponds to a stack space in the on-chip cache, and the target thread is any thread in the thread group.

[0062] When register overflow occurs during register allocation for the target thread, that is, when the number of physical registers of the target thread is insufficient, the overflowed virtual registers are written out to the target stack space corresponding to the target thread in the on-chip cache.

[0063] In the embodiments of this application, a corresponding target stack space is set for the target thread in the on-chip cache. Compared with the access speed of the video memory, the access speed of the on-chip cache is faster. Therefore, when register overflow occurs during register allocation for the target thread, the overflowed virtual registers are written out to the target stack space corresponding to the target thread in the on-chip cache, so that the access speed to the stack space is faster, thereby reducing the latency caused by register overflow and further improving the operator performance.

[0064] In some embodiments, after the related art writes the overflowed virtual registers to the video memory, during the execution of the operator, it is necessary to read back the overflowed virtual registers from the video memory for processing. Since the read and write speeds of the video memory are relatively slow, in this case, even if the number of register overflows is small, the additional read and write stack overhead caused by register overflow will cause a decrease in program performance, thereby affecting the operator performance.

[0065] In view of this, after this application writes the overflowed virtual registers to the target stack space corresponding to the target thread in the on-chip cache, during the execution of the operator, the overflowed virtual registers are read from the target stack space. Then, based on the read-back overflowed virtual registers, relevant calculations are performed to obtain the calculation result.

[0066] In the embodiments of this application, based on the high-speed read and write speed of the on-chip cache, the overflowed virtual registers are written to the on-chip cache during the compilation of the operator. When the data of the overflowed virtual registers is needed, the data is retrieved from the on-chip cache, thereby reducing the overhead of reading and writing the video memory and improving the running speed of the program.

[0067] In some embodiments, among the multiple physical registers associated with the target thread, there is included: a stack pointer register for pointing to the target stack space.

[0068] If a register spill occurs during the process of performing register allocation, the spilled virtual register is written out to the target stack space pointed to by the stack pointer register.

[0069] Specifically, one physical register is reserved from the multiple physical registers associated with the target thread as the stack pointer register, and this stack pointer register points to the target stack space corresponding to the target thread in the on-chip cache. The other physical registers associated with the target thread can be allocated to the virtual registers of the operator.

[0070] The stack pointer register stores the attribute information of the target stack space. In this way, when a register spill occurs in the register allocation for the target thread, based on the attribute information of the target stack space stored in the stack pointer register, the target stack space corresponding to the target thread can be quickly located in the on-chip cache, and the spilled virtual register is written out to the target stack space.

[0071] In addition, in the scenario where multiple threads in a thread group concurrently execute an operator, the target thread is one thread in the thread group, and each of the other threads in the thread group can reserve one physical register in the same way as the stack pointer register. When a register spill occurs, the spilled virtual register is written out to the stack space pointed to by the stack pointer register.

[0072] For example, see Figure 3 , the thread group includes multiple threads. For each thread, the thread corresponds to N physical registers, which are register 0, register 1,..., register (N - 1), and register N. Reserve register (N - 1) as the stack pointer register SP, and the stack pointer register SP is used to point to the stack space corresponding to this thread in the on-chip cache.

[0073] In the embodiments of this application, one physical register is reserved from the multiple physical registers associated with the target thread to point to the target stack space corresponding to the target thread in the on-chip cache. Therefore, when a register spill occurs in the register allocation of the target thread, the spilled virtual register can be quickly written out to the target stack space corresponding to the target thread, thereby reducing the latency caused by the register spill and further improving the performance of the operator.

[0074] In practical applications, the attribute information of the target stack space can be the starting address of the target stack space, the ending address of the target stack space, the address range of the target stack space, etc.

[0075] The following takes the attribute information of the target stack space being the starting address of the target stack space as an example for illustration:

[0076] Among the multiple physical registers associated with the target thread, there is a stack pointer register for pointing to the starting address of the target stack space.

[0077] Specifically, the starting address of the target stack space is also the starting physical address of the target stack space in the on-chip cache. If a register overflow occurs during the register allocation process, the overflowed virtual register is written out to the starting address of the target stack space pointed to by the stack pointer register.

[0078] In some embodiments, calculating the starting address of the target stack space consumes computing resources, and register overflow occurs relatively frequently. If the starting address of the target stack space is calculated every time a register overflow occurs, this will result in a large computing resource overhead and thus affect the operator performance.

[0079] In view of this, in the present application, if a register overflow occurs during the register allocation process and the register overflow occurs for the first time, the starting address of the target stack space is obtained and the starting address of the target stack space is saved in the stack pointer register. Then the overflowed virtual register is written out to the starting address of the target stack space pointed to by the stack pointer register.

[0080] When a register overflow occurs for the target thread subsequently, the overflowed virtual register can be directly written out to the target stack space pointed to by the stack pointer register according to the starting address of the target stack space in the stack pointer register, which avoids frequent calculation of the starting address of the target stack space, reduces the computing resource overhead, and thus improves the operator performance.

[0081] In some embodiments, the target thread is any one thread in the thread group. Each thread in the thread group corresponds to a stack space in the on-chip cache, and each thread corresponds to a number in the thread group. In this case, the embodiments of the present application at least obtain the starting address of the target stack space in the following manner:

[0082] Based on the initial address of the on-chip cache, the number of the target thread in the thread group, and the stack space size allocated for each thread, determine the starting address of the target stack space.

[0083] Specifically, a storage space (i.e., stack space) of size stackSize is pre-allocated for each thread in the on-chip cache. Then, based on the number of the target thread in the thread group and the stack space size allocated for each thread, determine the offset of the target stack space in the on-chip cache. Then, based on the initial address of the on-chip cache and the offset of the target stack space in the on-chip cache, obtain the starting address of the target stack space.

[0084] It should be noted that each thread in the thread group can obtain the starting address of the corresponding stack space by the above method, specifically as shown in the following formula (1):

[0085] SP = sm_base + threadIdx * stackSize …………(1)

[0086] Among them, sm_base is the initial address of the on-chip cache, threadIdx is the number of each thread in the thread group, and stackSize is the stack space size allocated for each thread.

[0087] In some embodiments, since the on-chip cache is not only used to save overflowed virtual registers but also needs to serve other functions of the artificial intelligence chip, there may be a situation where the storage space of the on-chip cache is insufficient.

[0088] In view of this, in this application, if a register overflow occurs during the process of performing register allocation and the storage space of the on-chip cache meets the register overflow requirement, the overflowed virtual registers are written out to the starting address of the target stack space pointed to by the stack pointer register.

[0089] That is to say, before writing the overflowed virtual registers to the on-chip cache, first determine whether the storage space of the on-chip cache is sufficient to save the overflowed virtual registers (that is, whether the storage space of the on-chip cache meets the register overflow requirement).

[0090] If the storage space of the on-chip cache is sufficient to save the overflowed virtual registers, the overflowed virtual registers are written out to the starting address of the target stack space pointed to by the stack pointer register.

[0091] In some embodiments, when the storage space of the on-chip cache is not sufficient to save the overflowed virtual registers, in order to avoid the loss of the overflowed virtual registers, this application proposes that if a register overflow occurs during the process of performing register allocation and the storage space of the on-chip cache does not meet the register overflow requirement, the overflowed virtual registers are written out to the video memory.

[0092] That is to say, when the storage space of the on-chip cache is not sufficient to save the overflowed virtual registers, the overflowed virtual registers are written out to the video memory, which avoids the loss of the overflowed virtual registers and ensures the integrity of the data.

[0093] For example, see Figure 4 , which is the flow of an optimized method for register overflow provided by an embodiment of this application. The flow of this method is executed by an artificial intelligence chip, and for each thread in the thread group, the following steps are respectively executed:

[0094] Step 401, reserve a physical register from multiple physical registers associated with the thread as the stack pointer register.

[0095] The stack pointer register is used to point to the stack space corresponding to the thread in the on-chip cache.

[0096] Step 402, during the operator compilation process, perform register allocation operations.

[0097] Step 403, determine whether register overflow occurs. If so, execute Step 404; otherwise, execute Step 409.

[0098] Step 404, determine whether it is the first occurrence of register overflow. If so, execute Step 405; otherwise, execute Step 406.

[0099] Step 405, calculate the starting address of the stack space pointed to by the stack pointer register, and then execute Step 406.

[0100] Step 406, determine whether the storage space of the on-chip cache is sufficient. If so, execute Step 407; otherwise, execute Step 408.

[0101] Step 407, write the overflowed virtual register to the stack space pointed to by the stack pointer register.

[0102] Specifically, write the variable associated with the overflowed virtual register to the stack space pointed to by the stack pointer register.

[0103] Step 408, write the overflowed virtual register to the video memory.

[0104] Specifically, write the variable associated with the overflowed virtual register to the video memory.

[0105] Step 409, allocate physical registers of the thread for multiple virtual registers of the operator.

[0106] In the embodiment of the present application, a corresponding stack space for the thread is set in the on-chip cache. Compared with the access speed of the video memory, the access speed of the on-chip cache is faster. Therefore, when register overflow occurs in the register allocation for the thread, the overflowed virtual register is written to the target stack space corresponding to the thread in the on-chip cache, so that the access speed to the stack space is faster, thereby reducing the latency caused by register overflow and further improving the operator performance.

[0107] Based on the same technical concept, the embodiment of the present application provides a structural schematic diagram of a register overflow optimization device, as Figure 5 shown. The register overflow optimization device 500 includes:

[0108] A register allocation module 501, configured to perform register allocation for the target thread executing the operator during the operator compilation process;

[0109] A write module 502, configured to write the overflowed virtual register to the target stack space corresponding to the target thread in the on-chip cache if a register overflow occurs during the execution of the register allocation process.

[0110] Optionally, it further includes a read module 503;

[0111] The read module 503 is specifically configured to:

[0112] Read the overflowed virtual register from the target stack space during the execution of the operator.

[0113] Optionally, among the multiple physical registers associated with the target thread, it includes: a stack pointer register for pointing to the target stack space;

[0114] The write module 502 is specifically configured to:

[0115] If a register overflow occurs during the execution of the register allocation process, write the overflowed virtual register to the target stack space pointed to by the stack pointer register.

[0116] Optionally, among the multiple physical registers associated with the target thread, it includes: a stack pointer register for pointing to the starting address of the target stack space;

[0117] The write module 502 is specifically configured to:

[0118] If a register overflow occurs during the execution of the register allocation process, write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register.

[0119] Optionally, the write module 502 is specifically configured to:

[0120] If a register overflow occurs during the execution of the register allocation process and the register overflow occurs for the first time, obtain the starting address of the target stack space and save the starting address of the target stack space in the stack pointer register;

[0121] Write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register.

[0122] Optionally, the target thread is any one of the threads in a thread group, and each thread in the thread group corresponds to a stack space in the on-chip cache;

[0123] The write module 502 is specifically configured to:

[0124] Determine the starting address of the target stack space based on the initial address of the on-chip cache, the number of the target thread in the thread group, and the stack space size allocated for each thread.

[0125] Optionally, the writing module 502 is specifically configured to:

[0126] If a register overflow occurs during the execution of the register allocation, and the storage space of the on-chip cache meets the register overflow requirement, write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register.

[0127] Optionally, the writing module 502 is further configured to:

[0128] If a register overflow occurs during the execution of the register allocation, and the storage space of the on-chip cache does not meet the register overflow requirement, write the overflowed virtual register to the video memory.

[0129] In the embodiment of the present application, a corresponding target stack space for the target thread is set in the on-chip cache. Compared with the access speed of the video memory, the access speed of the on-chip cache is faster. Therefore, when a register overflow occurs in the register allocation for the target thread, the overflowed virtual register is written to the target stack space corresponding to the target thread in the on-chip cache, so that the access speed to the stack space is faster, thereby reducing the latency caused by the register overflow and further improving the operator performance.

[0130] Based on the same technical concept, the embodiment of the present application provides a computer device, such as Figure 6 shown, including at least one artificial intelligence chip 100 and a memory 601 connected to the at least one artificial intelligence chip 100. In the embodiment of the present application, the specific connection medium between the artificial intelligence chip 100 and the memory 601 is not limited. Figure 6 Taking the example that the artificial intelligence chip 100 and the memory 601 are connected by a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0131] In the embodiment of the present application, the memory 601 stores instructions executable by the at least one artificial intelligence chip 100. The at least one artificial intelligence chip 100 can execute the steps of the above register overflow optimization method by executing the instructions stored in the memory 601.

[0132] Among them, the artificial intelligence chip 100 is the control center of the computer device. It can connect various parts of the computer device through various interfaces and circuits. By running or executing instructions stored in the memory 601 and calling data stored in the memory 601, register overflow optimization can be achieved. Optionally, the artificial intelligence chip 100 may include one or more processing units. The artificial intelligence chip 100 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 601 may be implemented on the same chip. In some embodiments, they may also be separately implemented on independent chips.

[0133] The artificial intelligence chip 100 may be a general-purpose processor, such as a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0134] The memory 601, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory 601 can include at least one type of storage medium. For example, it can include flash memory, hard disks, multimedia cards, card-type memories, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memories, magnetic disks, optical discs, and so on. The memory 601 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer device, but is not limited thereto. The memory 601 in the embodiments of the present application can also be a circuit or any other device capable of implementing a storage function, for storing program instructions and / or data.

[0135] Based on the same inventive concept, embodiments of the present application provide a computer-readable storage medium storing a computer program executable by a computer device. When the computer program runs on the computer device, it causes the computer device to execute the steps of the above register spill optimization method.

[0136] Based on the same inventive concept, embodiments of the present application provide a computer program product. The computer program product includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions that, when executed by a computer device, cause the computer device to execute the steps of the above register spill optimization method.

[0137] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, or a computer program product. Therefore, the present invention can take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0138] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and combinations of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to produce a machine, such that the instructions executed by the processors of the computer device or other programmable data processing device produce means for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0139] These computer program instructions can also be stored in a computer-readable memory that can direct a computer device or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufactured article including instruction means that implement the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0140] These computer program instructions can also be loaded onto a computer device or other programmable data processing device, such that a series of operation steps are executed on the computer device or other programmable device to produce a process implemented by the computer device, so that the instructions executed on the computer device or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0141] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0142] Obviously, those skilled in the art can make various changes and variations to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A register overflow optimization method, characterized in that, Including: For each thread in the thread group, allocate a stack space in the on-chip cache; the starting address of the stack space is the starting physical address of the stack space in the on-chip cache; multiple threads in the thread group execute operators concurrently; the target thread is any one thread in the thread group; the target thread corresponds to the target stack space in the on-chip cache. Based on the number of the target thread in the thread group and the size of the stack space allocated for each thread, determine the offset of the target stack space in the on-chip cache. Based on the initial address of the on-chip cache and the offset of the target stack space in the on-chip cache, obtain the starting address of the target stack space. During the operator compilation process, perform register allocation for the target thread that executes the operator; among the multiple physical registers associated with the target thread, there is a stack pointer register for pointing to the starting address of the target stack space. If a register overflow occurs during the process of performing the register allocation, write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register.

2. The method according to claim 1, characterized in that, Also including: During the process of executing the operator, read the overflowed virtual register from the target stack space.

3. The method according to claim 1, wherein The step of if a register overflow occurs during the process of performing the register allocation, write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register includes: If a register overflow occurs during the process of performing the register allocation and the register overflow occurs for the first time, obtain the starting address of the target stack space and save the starting address of the target stack space in the stack pointer register. Write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register.

4. The method according to claim 1, characterized in that, The step of if a register overflow occurs during the process of performing the register allocation, write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register includes: If a register overflow occurs during the process of performing the register allocation and the storage space of the on-chip cache meets the register overflow requirement, write the overflowed virtual register to the starting address of the target stack space pointed to by the stack pointer register.

5. The method according to claim 4, wherein Also including: If a register overflow occurs during the process of performing the register allocation and the storage space of the on-chip cache does not meet the register overflow requirement, write the overflowed virtual register to the video memory.

6. A computer device, comprising a memory, an artificial intelligence chip, and a computer program stored in the memory and executable on the artificial intelligence chip, characterized in that, When the artificial intelligence chip executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer device. When the computer program runs on the computer device, the computer device is caused to execute the steps of the method according to any one of claims 1 to 5.

8. A computer program product, characterized in that, The computer program product includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions that, when executed by a computer device, cause the computer device to perform the steps of the method according to any one of claims 1-5.

Images