A synchronous resource allocation method, device, storage medium and program product

Abstract

The embodiment of the present application provides a synchronization resource allocation method, device, storage medium and program product, which relates to the field of artificial intelligence technology. The method includes: according to the strong synchronization instructions in the kernel function of the operator, the kernel function is split into multiple instruction fragments, and then corresponding directed acyclic subgraphs are constructed for the multiple instruction fragments, wherein the constructed multiple directed acyclic subgraphs are independent of each other. Therefore, according to the directed acyclic subgraph of each instruction fragment, when allocating physical synchronization resources for each instruction fragment, the number of synchronization points is greatly reduced, which effectively reduces the situation of insufficient physical synchronization resources, thereby reducing the time consumption of synchronization resource allocation and improving the efficiency of operator compilation and execution. Secondly, by traversing the directed acyclic subgraph corresponding to each instruction fragment, physical synchronization resources are allocated to each instruction fragment, which greatly reduces the time complexity of traversal compared to traversing and analyzing the directed acyclic graph of the entire kernel function.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular, to a synchronous resource allocation method, device, storage medium, and program product. Background Art

[0002] Artificial intelligence models usually refer to neural network models that are trained for inference and prediction, such as image inference models, speech inference models, etc. The operations of artificial intelligence models can be implemented by operators in the computation graph. Operators refer to various operations performed on tensors at each layer in the artificial intelligence model. For example, the convolution operation performed by the convolution layer of the artificial intelligence model on the input data of the artificial intelligence model is the convolution operator.

[0003] During the operator execution process, multiple threads used to execute the operator need to be synchronized to ensure that data is generated and consumed in the correct timing, so physical synchronization resources are particularly important. However, the number of physical synchronization resources is limited and cannot be used indefinitely. Therefore, physical synchronization resources need to be allocated to operators during the operator compilation phase.

[0004] However, when the kernel function of an operator is too large, there are too many synchronization points in the kernel function, so there is often a shortage of physical synchronization resources, which leads to a long time spent on allocating synchronization resources, thus affecting the efficiency of operator compilation and execution. Summary of the invention

[0005] The embodiments of the present application provide a synchronous resource allocation method, device, storage medium and program product for reducing the time consumption of synchronous resource allocation and thereby improving the efficiency of operator compilation and execution.

[0006] On the one hand, an embodiment of the present application provides a synchronization resource allocation method, including:

[0007] According to the strong synchronization instructions in the kernel function of the operator, split the kernel function into multiple instruction fragments;

[0008] For the multiple instruction fragments, the following operations are performed respectively:

[0009] Construct a directed acyclic subgraph corresponding to an instruction fragment;

[0010] According to the directed acyclic subgraph corresponding to the one instruction fragment, physical synchronization resources are allocated to the one instruction fragment.

[0011] On the one hand, an embodiment of the present application provides a synchronous resource allocation device, including:

[0012] A splitting module, used for splitting the kernel function of the operator into multiple instruction fragments according to the strong synchronization instructions in the kernel function of the operator;

[0013] The processing module is used to perform the following operations on the multiple instruction fragments:

[0014] Construct a directed acyclic subgraph corresponding to an instruction fragment;

[0015] According to the directed acyclic subgraph corresponding to the one instruction fragment, physical synchronization resources are allocated to the one instruction fragment.

[0016] Optionally, the strong synchronization instruction is associated with a strong synchronization identifier, and the strong synchronization identifier is used to represent a synchronization point between multiple threads executing the kernel function.

[0017] Optionally, the processing module is further used for:

[0018] Corresponding physical synchronization resources are allocated to the synchronization point represented by the strong synchronization identifier.

[0019] Optionally, when multiple threads executing the kernel function are synchronized to the synchronization point represented by the strong synchronization identifier, multiple instructions in the kernel function that are located before the strong synchronization instruction have been executed, and the allocated physical synchronization resources have been recovered.

[0020] Optionally, the processing module is specifically used for:

[0021] Traversing multiple logical synchronization identifiers included in the one instruction fragment, each logical synchronization identifier is associated with a group of data production instructions and data consumption instructions in the one instruction fragment, the group of data production instructions and data consumption instructions are executed by different threads, and the logical synchronization identifier is used to represent the synchronization point between the different threads;

[0022] Based on the multiple logic synchronization identifiers and the execution logic of the one instruction fragment, a directed acyclic subgraph of the one instruction fragment is constructed.

[0023] Optionally, the processing module is specifically used for:

[0024] The directed acyclic subgraph of the one instruction fragment is traversed, and corresponding physical synchronization resources are sequentially allocated to the synchronization points represented by the multiple logical synchronization identifiers.

[0025] Optionally, the processing module is specifically used for:

[0026] The directed acyclic subgraph of the one instruction fragment is traversed, and corresponding physical synchronization resources are sequentially allocated to the synchronization points represented by the multiple logical synchronization identifiers based on the total synchronization resources.

[0027] Optionally, the processing module is further used for:

[0028] In the kernel function, the multiple logical synchronization identifiers are respectively modified into identification information of corresponding physical synchronization resources.

[0029] On the one hand, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned synchronous resource allocation method when executing the computer program.

[0030] On the one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program executable by a computer device. When the computer program runs on the computer device, the computer device executes the steps of the above-mentioned synchronous resource allocation method.

[0031] On the one hand, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned synchronous resource allocation method.

[0032] In the embodiment of the present application, according to the strong synchronization instructions in the kernel function of the operator, the kernel function is split into multiple instruction fragments, and then corresponding directed acyclic subgraphs are constructed for the multiple instruction fragments, wherein the constructed multiple directed acyclic subgraphs are independent of each other and do not affect each other. Therefore, according to the directed acyclic subgraph of each instruction fragment, when allocating physical synchronization resources to each instruction fragment, the number of synchronization points is greatly reduced, which effectively reduces the situation of insufficient physical synchronization resources, thereby reducing the time consumption of synchronization resource allocation and improving the efficiency of operator compilation and execution.

[0033] Secondly, this application allocates physical synchronization resources to each instruction fragment by traversing the directed acyclic subgraph corresponding to each instruction fragment. Compared with traversing and analyzing the directed acyclic graph of the entire kernel function, this greatly reduces the time complexity of traversal, thereby reducing the time consumption of synchronization resource allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0035] Figure 1 A schematic diagram of the structure of an artificial intelligence chip provided in an embodiment of the present application;

[0036] Figure 2 A schematic diagram of a synchronous resource allocation method provided in an embodiment of the present application Figure 1 ;

[0037] Figure 3 A schematic diagram of a directed acyclic subgraph provided in an embodiment of the present application Figure 1 ;

[0038] Figure 4 A schematic diagram of a synchronous resource allocation method provided in an embodiment of the present application Figure 2 ;

[0039] Figure 5 A schematic diagram of a directed acyclic subgraph provided in an embodiment of the present application Figure 2 ;

[0040] Figure 6 A schematic diagram of the structure of a synchronous resource allocation device provided in an embodiment of the present application;

[0041] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and beneficial effects of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] refer to Figure 1 , which is a structural diagram of an artificial intelligence chip applicable to an embodiment of the present application. The artificial intelligence chip 100 includes at least: a tensor core 101, a computing unit 102, a shared memory 103, and a register stack 104.

[0044] The artificial intelligence chip 100 includes a plurality of computing units 102, each of which is used to perform scalar and vector operations.

[0045] In some embodiments, each computing unit 102 includes: a plurality of single instruction multiple data (Single Instruction Multiple Data, SIMD for short) arithmetic logic units, or a plurality of single instruction multiple thread (Single Instruction Multiple Thread, SIMT for short) arithmetic logic units.

[0046] In some embodiments, each computing unit 102 includes: a plurality of SIMD arithmetic logic units and a plurality of SIMT arithmetic logic units.

[0047] The tensor core 101 is used to perform convolution operations or matrix multiplication. The artificial intelligence chip 100 can be used to execute various types of operators, including but not limited to: convolution operators, attention mechanism operators, and matrix multiplication operators.

[0048] The tensor core 101 and the computing unit 102 can exchange small blocks of data through a register file 104 with fast access speed but relatively limited capacity, and can also transfer large blocks of data through a shared memory 103 with relatively slow access speed but large capacity.

[0049] Whether the data is transferred between the tensor core 101 and the computing unit 102 through the register file 104 or through the shared memory 103, the tensor core 101 and the computing unit 102 need to be synchronized to ensure that the data is generated and consumed in the correct timing, so the physical synchronization resource is particularly important, wherein the physical synchronization resource can be a physical barrier resource. It should be noted that multiple threads inside the computing unit 102 or multiple threads inside the tensor core 101 also need to be synchronized through the physical synchronization resource to ensure that the data is generated and consumed in the correct timing.

[0050] According to the scope of physical synchronization resources, physical synchronization resources can be divided into the following two types of resources: thread group barrier (TG barrier for short) and wide thread group barrier (WTG barrier for short), where the thread group barrier is used for synchronization of the thread group inside the computing unit 102; the wide thread group barrier is used for synchronization between the thread group of the tensor core 101 and the thread groups of all computing units 102.

[0051] In addition to the above-mentioned structure, the artificial intelligence chip 100 in the present application may also include other structures, which are not specifically limited in the present application.

[0052] The artificial intelligence chip 100 may be: a graphics processing unit (GPU), a general-purpose computing on graphics processing units (GPGPU), a domain specific architecture (DSA), etc.

[0053] During the operator execution process, multiple threads used to execute the operator need to be synchronized to ensure that data is generated and consumed in the correct timing, so physical synchronization resources are particularly important. However, the number of physical synchronization resources is limited and cannot be used indefinitely. Therefore, physical synchronization resources need to be allocated to operators during the operator compilation phase.

[0054] However, when the kernel function of an operator is too large, there are too many synchronization points in the kernel function, so there is often a shortage of physical synchronization resources, which leads to a long time spent on allocating synchronization resources, thus affecting the efficiency of operator compilation and execution.

[0055] In view of this, the present application provides a process of a synchronous resource allocation method, such as Figure 2 As shown, the process of the method is executed by a computer device and includes the following steps:

[0056] Step 201 : split the kernel function into multiple instruction fragments according to the strong synchronization instructions in the kernel function of the operator.

[0057] Specifically, the kernel function of an operator refers to the program code that the operator executes on the artificial intelligence chip. The operator in this application can be Figure 1 The artificial intelligence chip 100 shown executes. Before the artificial intelligence chip executes the operator, the kernel function of the operator is first compiled by the computer device, and the synchronous resource allocation method in the present application is executed during the operator compilation process.

[0058] In the present application, the kernel function of the operator includes one or more strong synchronization instructions, and the kernel function is split into multiple instruction fragments with the strong synchronization instructions in the kernel function as the dividing point, and each instruction fragment includes at least one instruction. It should be noted that the strong synchronization instruction can be an instruction in a certain instruction fragment (an instruction fragment before the strong synchronization instruction, or an instruction fragment after the strong synchronization instruction), or it can be independent of each instruction fragment, and this application does not make specific limitations on this.

[0059] For example, assuming that a kernel function includes a strong synchronization instruction, the kernel function is split into two instruction fragments using the strong synchronization instruction as a dividing point, and each instruction fragment includes multiple instructions.

[0060] In some embodiments, the strong synchronization instruction is associated with a strong synchronization identifier, and the strong synchronization identifier is used to represent a synchronization point between multiple threads that execute the kernel function.

[0061] Specifically, in the kernel function of the operator, the strong synchronization instruction is associated with a strong synchronization identifier, which may be a logical synchronization resource number, for example, a logical fence resource number (ie, a logical bar id).

[0062] A synchronization point refers to an execution position during the execution of an operator, and multiple threads are synchronized at this execution position. In this application, a synchronization point represented by a strong synchronization identifier refers to an execution position of a strong synchronization instruction during the execution of an operator, and all threads executing the kernel function achieve thread synchronization at this execution position. In other words, when all threads executing the kernel function have executed the strong synchronization instruction, all threads can continue to execute backwards.

[0063] A corresponding physical synchronization resource is allocated to each synchronization point represented by a strong synchronization identifier, wherein the physical synchronization resource may be a physical fence resource.

[0064] In the specific implementation, the physical fence resource maintains a counter and a synchronization point inside, where the value recorded by the counter is equal to the number of threads executing the kernel function. When a thread executing the kernel function executes a strong synchronization instruction (i.e., the synchronization point represented by the strong synchronization identifier associated with the strong synchronization instruction), the physical fence resource will block the thread to wait for other threads to arrive, and at the same time, the counter in the physical fence resource will be reduced by 1. When all threads execute the strong synchronization instruction, at this time, the counter is 0, the fence is opened, and all threads continue to execute backward to ensure that multiple threads executing the kernel function are synchronized to the synchronization point represented by the strong synchronization identifier.

[0065] In some embodiments, when multiple threads executing a kernel function are synchronized to a synchronization point represented by a strong synchronization identifier, multiple instructions in the kernel function that precede the strong synchronization instruction have all been executed, and the allocated physical synchronization resources have all been reclaimed. Therefore, each instruction fragment can independently allocate physical synchronization resources, and the physical resources are completely isolated.

[0066] Step 202: for multiple instruction fragments, execute the following steps 2021 and 2022 respectively:

[0067] Specifically, for each instruction fragment, when the instruction fragment is executed, multiple instructions before the instruction fragment have been executed, and the allocated physical synchronization resources have been recovered. Therefore, each instruction fragment can independently allocate physical synchronization resources, and there will be no resource conflict with other instruction fragments during the physical synchronization resource allocation process.

[0068] In practical applications, physical synchronization resources can also be allocated to multiple instruction fragments in parallel. Since multiple instruction fragments can be allocated physical synchronization resources in parallel and independently, the time consumption of synchronization resource allocation is further reduced, thereby improving the efficiency of operator compilation and execution.

[0069] Step 2021, construct a directed acyclic subgraph corresponding to the instruction fragment.

[0070] Specifically, a directed acyclic graph is a directed graph without loops. Since the kernel function is split into multiple instruction fragments using strong synchronization instructions, after constructing a directed acyclic subgraph for each instruction fragment, multiple instruction fragments correspond to directed acyclic subgraphs, which can form a directed acyclic graph of the kernel function.

[0071] In some embodiments, multiple logical synchronization identifiers included in the traversal instruction fragment are each associated with a group of data production instructions and data consumption instructions in the instruction fragment, and the group of data production instructions and data consumption instructions are executed by different threads. The logical synchronization identifier is used to characterize the synchronization point between different threads. Based on the multiple logical synchronization identifiers and the execution logic of the instruction fragment, a directed acyclic subgraph of the instruction fragment is constructed.

[0072] Specifically, the logical synchronization identifier can be a logical synchronization resource number. For example, the logical synchronization identifier is a logical fence resource number (i.e., a logical bar id). In each group of data production instructions and data consumption instructions, the data production instructions and the data consumption instructions correspond to the same logical synchronization identifier. The data production instructions and the data consumption instructions are executed by different threads, and the data consumption instructions can only be executed after the data production instructions are executed.

[0073] The synchronization point between different threads executing data production instructions and data consumption instructions refers to: the execution position after the first thread (used to execute data production instructions) executes the data production instruction and before the second thread (used to execute data consumption instructions) executes the data consumption instruction; the first thread and the second thread are synchronized at this execution position.

[0074] The execution logic of the instruction fragment includes but is not limited to: sequential execution, branch execution, and loop execution.

[0075] It should be noted that in a set of data production instructions and data consumption instructions: the number of data production instructions and the number of data consumption instructions are not limited to one, but may be multiple. Accordingly, different threads executing a set of data production instructions and data consumption instructions are not limited to: the first thread and the second thread, but may be more threads, and this application does not make specific limitations on this.

[0076] For example, see Figure 3 The instruction fragment includes: data production instruction P0, data production instruction P1, data consumption instruction C2, data production instruction P3, and data consumption instruction C4 executed by thread 1; and data consumption instruction C0, data production instruction P2, data consumption instruction C1, data production instruction P4, and data consumption instruction C3 executed by thread 2.

[0077] In the instruction fragment, the data production instruction P0 and the data consumption instruction C0 are associated with the logical synchronization identifier L0; the data production instruction P1 and the data consumption instruction C1 are associated with the logical synchronization identifier L1; the data production instruction P2 and the data consumption instruction C2 are associated with the logical synchronization identifier L2; the data production instruction P3 and the data consumption instruction C3 are associated with the logical synchronization identifier L3; the data production instruction P4 and the data consumption instruction C4 are associated with the logical synchronization identifier L4.

[0078] Based on the above four logical synchronization flags in the instruction fragment and the execution logic of the instruction fragment (sequential execution), a directed acyclic subgraph of the instruction fragment is constructed, as shown below: Figure 3 As shown, the nodes in the directed acyclic subgraph are data production instructions or data consumption instructions, and the direction in the directed acyclic subgraph represents the order in which the instructions are executed.

[0079] The logical synchronization identifier L0 specifies the synchronization point between thread 1 executing the data production instruction P0 and thread 2 executing the data consumption instruction C0; that is, thread 2 needs to wait until thread 1 completes executing the data production instruction P0 before starting to execute the data consumption instruction C0. Therefore, in the directed acyclic subgraph, the data production instruction P0 points to the data consumption instruction C0.

[0080] In the same way, the data production instruction P1 can point to the data consumption instruction C1, the data production instruction P2 can point to the data consumption instruction C2, the data production instruction P3 can point to the data consumption instruction C3, and the data production instruction P4 can point to the data consumption instruction C4.

[0081] In addition, in the directed acyclic subgraph, multiple instructions on thread 1 or thread 2 are executed in sequence.

[0082] It should be noted that the threads that execute instruction fragments in the present application are not limited to two threads. Therefore, the number of threads in the directed acyclic subgraph is not limited to two, but can also be other numbers of threads. The present application does not make any specific limitation on this.

[0083] In addition, when the instruction fragment contains a strong synchronization instruction, the instruction fragment also contains a strong synchronization identifier. At this time, a directed acyclic subgraph of the instruction fragment can be constructed based on the strong synchronization identifier, multiple logical synchronization identifiers and the execution logic of the instruction fragment. The specific construction process is the same as the previous article and will not be repeated here.

[0084] Step 2022: Allocate physical synchronization resources for the instruction fragment according to the directed acyclic subgraph of the instruction fragment.

[0085] Specifically, when the kernel function of the operator is too large, the directed acyclic graph corresponding to the kernel function will be too large. When traversing the entire directed acyclic graph of the kernel function and allocating physical synchronization resources for the kernel function, the traversal takes a long time. For example, if the directed acyclic graph of the kernel function contains n nodes, the time complexity of analyzing and traversing the directed acyclic graph is , the larger n is, the more time-consuming it is.

[0086] In view of this, in this application, a directed acyclic subgraph is constructed for each instruction fragment, which greatly reduces the number of nodes in the directed acyclic subgraph. Then, the directed acyclic subgraph of each instruction fragment is traversed, and physical synchronization resources are allocated to each instruction fragment, thereby reducing the time consumption of analyzing and traversing the directed acyclic graph.

[0087] For example, the directed acyclic graph of the kernel function includes n nodes. Divide the kernel function into x instruction fragments and construct a corresponding directed acyclic subgraph for each instruction fragment to obtain a directed acyclic subgraph. Figure 1 , directed acyclic Figure 2 , ..., directed acyclic subgraph x, where the directed acyclic subgraph Figure 1 Including n1 nodes, directed acyclic Figure 2 Including n2 nodes, ..., directed acyclic subgraph x includes nx nodes, where (n1 + n2 + ... + nx) = n. At this time, when analyzing and traversing x directed acyclic subgraphs, the time complexity is .

[0088] Compared to complexity For example, when analyzing and traversing x directed acyclic subgraphs in this application, the time complexity is much lower.

[0089] In some embodiments, the directed acyclic subgraph of the instruction fragment is traversed, and corresponding physical synchronization resources are sequentially allocated to the synchronization points represented by multiple logical synchronization identifiers.

[0090] Specifically, the methods for traversing the directed acyclic subgraph include: depth-first traversal and breadth-first traversal, etc. By traversing the directed acyclic subgraph of the instruction fragment, physical synchronization resources are allocated to each synchronization point represented by the logical synchronization identifier in turn. In the resource allocation process, the mapping relationship between the logical synchronization identifier and the allocated physical synchronization resource is recorded. After the corresponding physical synchronization resource is allocated to the synchronization point represented by each logical synchronization identifier, multiple logical synchronization identifiers are respectively modified into the identification information of the corresponding physical synchronization resources in the kernel function.

[0091] In some embodiments, when an instruction fragment includes a strong synchronization instruction, traversing the directed acyclic subgraph of the instruction fragment can not only allocate corresponding physical synchronization resources to the synchronization points represented by multiple logical synchronization identifiers in sequence, but also allocate corresponding physical synchronization resources to the synchronization points represented by the strong synchronization identifier.

[0092] In addition, the present application not only modifies multiple logical synchronization identifiers into identification information of corresponding physical synchronization resources, but also modifies each strong synchronization identifier into identification information of corresponding physical synchronization resources in the kernel function.

[0093] In this way, during the operator execution phase, when the operator executes to a synchronization point specified by a logical synchronization identifier or a synchronization point specified by a strong synchronization identifier, the physical synchronization resource can be called based on the identification information of the corresponding physical synchronization resource to achieve thread synchronization.

[0094] In some embodiments, the directed acyclic subgraph of the instruction fragments is traversed, and corresponding physical synchronization resources are sequentially allocated to synchronization points represented by multiple logical synchronization identifiers based on the total synchronization resources.

[0095] Specifically, total synchronization resources refer to: all physical synchronization resources allocated to operators in the artificial intelligence chip. Since the kernel function is split into multiple instruction fragments based on the strong synchronization instructions in the kernel function, when each instruction fragment is executed, the multiple instructions before the instruction fragment have been executed, and the allocated physical synchronization resources have been recovered. Therefore, each instruction fragment can independently allocate physical synchronization resources based on the total synchronization resources, and there will be no resource conflict with other instruction fragments during the physical synchronization resource allocation process.

[0096] During the resource allocation process, not only the mapping relationship between the logical synchronization identifier and the allocated physical synchronization resources is recorded, but also the remaining available physical synchronization resources are recorded, wherein the remaining available physical synchronization resources include: physical synchronization resources that are not allocated in the total synchronization resources and physical synchronization resources reused from the total synchronization resources.

[0097] In some embodiments, after allocating corresponding physical synchronization resources to the synchronization point represented by each logical synchronization identifier, it is determined whether the remaining available physical synchronization resources are greater than or equal to zero. If greater than or equal to zero, it means that the physical synchronization resources are sufficient, so multiple logical synchronization identifiers are respectively modified to the identification information of the corresponding physical synchronization resources in the kernel function. If less than zero, it means that the physical synchronization resources are insufficient, so the program reports an error and exits.

[0098] In some embodiments, since the kernel function is split into multiple instruction fragments based on the strong synchronization instruction in the kernel function as a dividing point, after each instruction fragment (except the last instruction fragment) is executed, a strong synchronization instruction (corresponding to the strong synchronization identifier) ​​will be executed next. Then, for each instruction fragment, physical synchronization resources can be allocated to the instruction fragment from the total synchronization resources, and then physical synchronization resources can be allocated to the strong synchronization instruction to be executed next from the remaining available physical synchronization resources. Then, it is determined whether the physical synchronization resources are sufficient; if sufficient, the multiple logical synchronization identifiers contained in the instruction fragment and the strong synchronization identifier corresponding to the strong synchronization instruction in the kernel function are modified to the identification information of the corresponding physical synchronization resources; if insufficient, the program reports an error and exits.

[0099] In some embodiments, when the instruction fragment includes a strong synchronization instruction, the directed acyclic subgraph of the instruction fragment is traversed, and corresponding physical synchronization resources are sequentially allocated to the synchronization points represented by multiple logical synchronization identifiers based on the total synchronization resources, and corresponding physical synchronization resources are allocated to the synchronization points represented by the strong synchronization identifier. The specific method of allocating physical synchronization resources has been introduced in the previous text and will not be repeated here.

[0100] For example, see Figure 4 , a process of a synchronization resource allocation method provided in an embodiment of the present application, comprising the following steps:

[0101] Step 401 , dividing the kernel function of the operator into instruction fragment 1 and instruction fragment 2 with the strong synchronization instruction as the dividing point.

[0102] Step 402, traverse multiple logical synchronization identifiers included in instruction fragment 1, and construct a directed acyclic sub-chain based on multiple logical synchronization identifiers and the execution logic of instruction fragment 1. Figure 1 .

[0103] Step 403, traverse the directed acyclic sub Figure 1 , corresponding physical barrier resources are allocated in sequence to the synchronization points represented by multiple logical synchronization identifiers in the instruction fragment 1.

[0104] Specifically, see Figure 5 , directed acyclic Figure 1 It includes: thread 1 sequentially executes data production instruction P0, data production instruction P1, and data consumption instruction C2; and thread 2 sequentially executes data consumption instruction C0, data production instruction P2, and data consumption instruction C1.

[0105] In addition, in the directed acyclic Figure 1 In the example, data production instruction P0 points to data consumption instruction C0, data production instruction P1 points to data consumption instruction C1, and data production instruction P2 points to data consumption instruction C2.

[0106] Taking data production instruction P0 as the starting point, the directed acyclic Figure 1 When traversing to the data production instruction P0 pointing to the data consumption instruction C0, since both the data production instruction P0 and the data consumption instruction C0 correspond to the logical synchronization identifier L0, a physical fence resource is allocated to the synchronization point represented by the logical synchronization identifier L0. The physical fence resource is used to control thread 1 to execute the data production instruction P0 before thread 2 starts to execute the data consumption instruction C0.

[0107] When traversing to the data production instruction P1 pointing to the data consumption instruction C1, since both the data production instruction P1 and the data consumption instruction C1 correspond to the logical synchronization identifier L1, a physical fence resource is allocated to the synchronization point represented by the logical synchronization identifier L1. The physical fence resource is used to control thread 1 to execute the data production instruction P1 before thread 2 starts executing the data consumption instruction C1.

[0108] When traversing to the data production instruction P2 pointing to the data consumption instruction C2, since both the data production instruction P2 and the data consumption instruction C2 correspond to the logical synchronization identifier L2, a physical fence resource is allocated to the synchronization point represented by the logical synchronization identifier L2. The physical fence resource is used to control thread 2 to execute the data production instruction P2 before thread 1 starts executing the data consumption instruction C2.

[0109] Next, a physical fence resource is allocated to the strong synchronization instruction (ie, the synchronization point represented by the associated strong synchronization identifier). The physical fence resource is used to control the execution of instruction fragment 2 only when both thread 1 and thread 2 execute the strong synchronization instruction.

[0110] Step 404, traverse the multiple logical synchronization identifiers included in the instruction fragment 2, and construct a directed acyclic sub-chain based on the multiple logical synchronization identifiers and the execution logic of the instruction fragment 2. Figure 2 .

[0111] Step 405, traverse the directed acyclic sub Figure 2 , corresponding physical barrier resources are allocated in sequence to the synchronization points represented by multiple logical synchronization identifiers in the instruction fragment 2.

[0112] Specifically, see Figure 5 , thread 1 and thread 2 execute instruction fragment 1 first, and only when thread 1 and thread 2 both execute the strong synchronization instruction will they start executing instruction fragment 2.

[0113] The directed acyclic submodule corresponding to instruction fragment 2 Figure 2 It includes: data production instructions P3 and data consumption instructions C4 executed in sequence by thread 1; and data production instructions P4 and data consumption instructions C3 executed in sequence by thread 2.

[0114] In addition, in the directed acyclic Figure 2 In the example, data production instruction P3 points to data consumption instruction C3, and data production instruction P4 points to data consumption instruction C4.

[0115] Taking data production instruction P3 as the starting point, the directed acyclic Figure 2 When traversing to the data production instruction P3 pointing to the data consumption instruction C3, since both the data production instruction P3 and the data consumption instruction C3 correspond to the logical synchronization identifier L3, a physical fence resource is allocated to the synchronization point represented by the logical synchronization identifier L3. The physical fence resource is used to control thread 1 to execute the data production instruction P3 before thread 2 starts to execute the data consumption instruction C3.

[0116] When traversing to the data production instruction P4 pointing to the data consumption instruction C4, since both the data production instruction P4 and the data consumption instruction C4 correspond to the logical synchronization identifier L4, a physical fence resource is allocated to the synchronization point represented by the logical synchronization identifier L4. The physical fence resource is used to control thread 2 to execute the data production instruction P4 before thread 1 starts executing the data consumption instruction C4.

[0117] Step 406, after allocating physical barrier resources to instruction fragment 1 and instruction fragment 2, the program exits.

[0118] In the embodiment of the present application, according to the strong synchronization instructions in the kernel function of the operator, the kernel function is split into multiple instruction fragments, and then corresponding directed acyclic subgraphs are constructed for the multiple instruction fragments, wherein the constructed multiple directed acyclic subgraphs are independent of each other and do not affect each other. Therefore, according to the directed acyclic subgraph of each instruction fragment, when allocating physical synchronization resources to each instruction fragment, the number of synchronization points is greatly reduced, which effectively reduces the situation of insufficient physical synchronization resources, thereby reducing the time consumption of synchronization resource allocation and improving the efficiency of operator compilation and execution.

[0119] Secondly, this application allocates physical synchronization resources to each instruction fragment by traversing the directed acyclic subgraph corresponding to each instruction fragment. Compared with traversing and analyzing the directed acyclic graph of the entire kernel function, this greatly reduces the time complexity of traversal, thereby reducing the time consumption of synchronization resource allocation.

[0120] Based on the same technical concept, the embodiment of the present application provides a structural diagram of a synchronous resource allocation device, such as Figure 6 As shown, the synchronous resource allocation device 600 includes:

[0121] A splitting module 601 is used to split the kernel function of the operator into multiple instruction fragments according to the strong synchronization instructions in the kernel function of the operator;

[0122] The processing module 602 is configured to respectively perform the following operations on the multiple instruction fragments:

[0123] Construct a directed acyclic subgraph corresponding to an instruction fragment;

[0124] According to the directed acyclic subgraph corresponding to the one instruction fragment, physical synchronization resources are allocated to the one instruction fragment.

[0125] Optionally, the strong synchronization instruction is associated with a strong synchronization identifier, and the strong synchronization identifier is used to represent a synchronization point between multiple threads executing the kernel function.

[0126] Optionally, when multiple threads executing the kernel function are synchronized to the synchronization point represented by the strong synchronization identifier, multiple instructions in the kernel function that are located before the strong synchronization instruction have been executed, and the allocated physical synchronization resources have been recovered.

[0127] Optionally, the processing module 602 is specifically configured to:

[0128] Traversing multiple logical synchronization identifiers included in the one instruction fragment, each logical synchronization identifier is associated with a group of data production instructions and data consumption instructions in the one instruction fragment, the group of data production instructions and data consumption instructions are executed by different threads, and the logical synchronization identifier is used to represent the synchronization point between the different threads;

[0129] Based on the multiple logic synchronization identifiers and the execution logic of the one instruction fragment, a directed acyclic subgraph of the one instruction fragment is constructed.

[0130] Optionally, the processing module 602 is specifically configured to:

[0131] The directed acyclic subgraph of the one instruction fragment is traversed, and corresponding physical synchronization resources are sequentially allocated to the synchronization points represented by the multiple logical synchronization identifiers.

[0132] Optionally, the processing module 602 is specifically configured to:

[0133] The directed acyclic subgraph of the one instruction fragment is traversed, and corresponding physical synchronization resources are sequentially allocated to the synchronization points represented by the multiple logical synchronization identifiers based on the total synchronization resources.

[0134] Optionally, the processing module 602 is further configured to:

[0135] In the kernel function, the multiple logical synchronization identifiers are respectively modified into identification information of corresponding physical synchronization resources.

[0136] Optionally, the processing module 602 is further configured to:

[0137] Corresponding physical synchronization resources are allocated to the synchronization point represented by the strong synchronization identifier.

[0138] In the embodiment of the present application, according to the strong synchronization instructions in the kernel function of the operator, the kernel function is split into multiple instruction fragments, and then corresponding directed acyclic subgraphs are constructed for the multiple instruction fragments, wherein the constructed multiple directed acyclic subgraphs are independent of each other and do not affect each other. Therefore, according to the directed acyclic subgraph of each instruction fragment, when allocating physical synchronization resources to each instruction fragment, the number of synchronization points is greatly reduced, which effectively reduces the situation of insufficient physical synchronization resources, thereby reducing the time consumption of synchronization resource allocation and improving the efficiency of operator compilation and execution.

[0139] Secondly, this application allocates physical synchronization resources to each instruction fragment by traversing the directed acyclic subgraph corresponding to each instruction fragment. Compared with traversing and analyzing the directed acyclic graph of the entire kernel function, this greatly reduces the time complexity of traversal, thereby reducing the time consumption of synchronization resource allocation.

[0140] Based on the same technical concept, the embodiment of the present application provides a computer device, such as Figure 7 As shown, it includes at least one processor 701 and a memory 702 connected to the at least one processor 701. The specific connection medium between the processor 701 and the memory 702 is not limited in the embodiment of the present application. Figure 7 For example, the processor 701 and the memory 702 are connected via a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0141] In the embodiment of the present application, the memory 702 stores instructions that can be executed by at least one processor 701, and the at least one processor 701 can perform the steps of the above-mentioned synchronization resource allocation method by executing the instructions stored in the memory 702.

[0142] Among them, the processor 701 is the control center of the computer device, and can use various interfaces and lines to connect various parts of the computer device, and realize synchronous resource allocation by running or executing instructions stored in the memory 702 and calling data stored in the memory 702. Optionally, the processor 701 may include one or more processing units, and the processor 701 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 701. In some embodiments, the processor 701 and the memory 702 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0143] The processor 701 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0144] The memory 702 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 702 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, and the like. The memory 702 is any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer device, but is not limited thereto. The memory 702 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0145] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a computer device. When the computer program runs on the computer device, the computer device executes the steps of the above-mentioned synchronous resource allocation method.

[0146] Based on the same inventive concept, an embodiment of the present application provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned synchronization resource allocation method.

[0147] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer device or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0149] These computer program instructions may also be stored in a computer readable memory capable of directing a computer device or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer device or other programmable data processing device so that a series of operation steps are executed on the computer device or other programmable device to produce a process implemented by the computer device, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0151] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0152] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A synchronous resource allocation method, characterized in that: include: During operator compilation, the kernel function of the operator is split into multiple instruction fragments according to the strong synchronization instruction in the kernel function of the operator, and the strong synchronization instruction is associated with a strong synchronization identifier, and the strong synchronization identifier is used to represent the synchronization point between multiple threads executing the kernel function; For the multiple instruction fragments, the following operations are performed respectively: Construct a directed acyclic subgraph corresponding to an instruction fragment; Traversing the directed acyclic subgraph corresponding to the one instruction fragment, allocating physical synchronization resources to synchronization points represented by multiple logical synchronization identifiers contained in the one instruction fragment based on total synchronization resources, wherein the total synchronization resources refer to: all physical synchronization resources allocated to the operator in the artificial intelligence chip; When multiple threads executing the kernel function are synchronized to the synchronization point represented by the strong synchronization identifier, multiple instructions in the kernel function that are located before the strong synchronization instruction have been executed, and the allocated physical synchronization resources have been recovered.

2. The method according to claim 1, characterized in that Also includes: Corresponding physical synchronization resources are allocated to the synchronization point represented by the strong synchronization identifier.

3. The method according to claim 1, characterized in that The step of constructing a directed acyclic subgraph corresponding to an instruction fragment includes: Traversing multiple logical synchronization identifiers included in the one instruction fragment, each logical synchronization identifier is associated with a group of data production instructions and data consumption instructions in the one instruction fragment, the group of data production instructions and data consumption instructions are executed by different threads, and the logical synchronization identifier is used to represent the synchronization point between the different threads; Based on the multiple logic synchronization identifiers and the execution logic of the one instruction fragment, a directed acyclic subgraph corresponding to the one instruction fragment is constructed.

4. The method according to claim 1, characterized in that Also includes: In the kernel function, the multiple logical synchronization identifiers are respectively modified into identification information of corresponding physical synchronization resources.

5. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium, characterized in that: It stores a computer program executable by a computer device, and when the computer program is run on the computer device, the computer device executes the steps of the method as claimed in any one of claims 1 to 4.

7. A computer program product, characterized in that The computer program product comprises a computer program stored on a computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer device, the computer device is caused to execute the steps of the method according to any one of claims 1 to 4.

Images