A resource allocation method, apparatus and medium

Abstract

Embodiments of the present application disclose a resource allocation method, device and medium, wherein the resource allocation method is applied to a stream multiprocessor running N thread blocks, and the resource allocation method comprises: in response to receiving a thread queue mutual exclusion request set, based on thread identifiers corresponding to the thread queue mutual exclusion request set, adjusting request information in at least one thread queue mutual exclusion tracker matched with the thread queue mutual exclusion request set by using a mutual exclusion lock mechanism, to obtain an adjusted at least one thread queue mutual exclusion tracker; in the adjusted at least one thread queue mutual exclusion tracker, determining a target tracker satisfying a resource allocation condition; and based on a distribution request corresponding to a target thread identifier in the target tracker, distributing operation resources associated with the N thread blocks to a thread queue associated with the target thread identifier. In this way, the stream multiprocessor can provide stronger computing power support.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a resource allocation method, apparatus and medium. Background Technology

[0002] In related technologies, with the widespread application of artificial intelligence, matrix operations have become an increasingly common and urgent need. Typically, an Arithmetic and Logic Unit (ALU) for matrix multiplication is added to a traditional Graphics Processing Unit (GPU). This reuses the memory bandwidth and general-purpose computing unit parallelism of the original GPU, providing input data movement and additional auxiliary computation for matrix operations. However, if the size of the matrix multiplier continues to increase, when dealing with larger matrices, it is usually limited by the hierarchical structure, memory bandwidth, and the size of the general-purpose computing units in the original GPU. This leads to complex management issues during resource allocation, resulting in some input matrix operations not being efficiently moved to the input buffer of the matrix multiplier. Summary of the Invention

[0003] This application provides a resource allocation technical solution.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a resource allocation method applied to a stream multiprocessor running N thread blocks. The resource allocation method includes:

[0006] In response to receiving a set of thread queue mutex requests, based on the thread identifiers corresponding to the set of thread queue mutex requests, the request information in at least one thread queue mutex tracker matching the set of thread queue mutex requests is adjusted using a mutex lock mechanism to obtain at least one adjusted thread queue mutex tracker. The set of thread queue mutex requests includes: mutual exclusion requests from M threads belonging to different thread queues sent by each thread block. Each thread queue consists of N sequentially connected threads, and the N threads correspond to N thread blocks.

[0007] In at least one adjusted thread queue mutex tracker, determine the target tracker that meets the resource allocation conditions;

[0008] Based on the allocation request corresponding to the target thread identifier within the target tracker, the computing resources associated with N thread blocks are allocated to the thread queue associated with the target thread identifier.

[0009] This application embodiment provides a resource allocation device, which includes:

[0010] The first processing module is used to respond to the received thread queue mutex request set, and based on the thread identifier corresponding to the thread queue mutex request set, adjust the request information in at least one thread queue mutex tracker that matches the thread queue mutex request set using a mutex lock mechanism to obtain at least one adjusted thread queue mutex tracker; the thread queue mutex request set includes: mutual exclusion requests sent by each thread block to M threads belonging to different thread queues, each thread queue consisting of N sequentially consecutive threads, and the N threads corresponding to N thread blocks respectively;

[0011] The second processing module is used to determine the target tracker that meets the resource allocation conditions among at least one adjusted thread queue mutex tracker.

[0012] The third processing module is used to allocate the computing resources associated with N thread blocks to the thread queue associated with the target thread identifier based on the allocation request corresponding to the target thread identifier in the target tracker.

[0013] Correspondingly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed, can implement the above-described resource allocation method.

[0014] In this embodiment, the resource allocation method described above adjusts the thread queue mutex tracker based on the thread queue mutex request set using a mutex lock mechanism. This ensures that N sequentially connected threads within the same thread group and thread queue can exclusively access the relevant resources during resource allocation, thanks to the mutex lock mechanism. Specifically, after determining the target tracker that meets the resource allocation conditions, the computational resources associated with the N thread blocks are allocated to the thread queue associated with the target thread identifier. This allows the N sequentially connected threads running within the streaming multiprocessor to exclusively access the hardware computational resources associated with the N thread blocks. Furthermore, when these N sequentially connected threads exclusively access the hardware computational resources (taking matrix operation units as an example), it is equivalent to concatenating the matrix operation units associated with each of the N thread blocks. Because these threads are sequentially connected and within the same thread group and thread queue, they can perform computational processing synchronously, thereby enabling the processing of larger matrices. Simultaneously, these threads running synchronously can share the input data of the hardware computational resources associated with the N thread blocks. This synchronous operation and data sharing mechanism enables the computing resources within the streaming multiprocessor to be synchronized for computation and data transfer. When processing partial input matrix operations, this synchronization mechanism allows data to be moved more efficiently to the input buffer of the matrix multiplier, thereby providing stronger computing power support for the streaming multiprocessor.

[0015] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the technical solutions provided in the embodiments of this application. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0017] Figure 1 A flowchart illustrating a resource allocation method provided in this application embodiment. Figure 1 ;

[0018] Figure 2 A flowchart illustrating a resource allocation method provided in this application embodiment. Figure 2 ;

[0019] Figure 3 A flowchart illustrating a resource allocation method provided in this application embodiment. Figure 3 ;

[0020] Figure 4 A flowchart illustrating a resource allocation method provided in this application embodiment. Figure 4 ;

[0021] Figure 5 A schematic diagram of the system allocation structure corresponding to the resource allocation method provided in the embodiments of this application;

[0022] Figure 6 This is a schematic diagram of the structural composition of the resource allocation device provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of the invention will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate the embodiments of this application, but are not intended to limit the scope of the embodiments of this application.

[0024] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0025] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which embodiments of this application belong. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of embodiments of this application.

[0027] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0028] 1) Mutex lock: Used to protect operations on shared resources, ensuring that the code of a thread operating on a shared resource can execute completely without being interrupted by other threads. The code segment operating on a shared resource is usually called the critical section, and the shared resource itself can be called a critical resource. Therefore, the working principle of a mutex lock is to lock the critical section, ensuring that the thread in the critical section is not interrupted by other threads, thus ensuring the complete execution of the critical section.

[0029] 2) Streaming multiprocessors (SMs) are the core modules that build the entire GPU (executing the entire Kernel Grid). A single streaming multiprocessor typically runs multiple thread blocks simultaneously. Each streaming multiprocessor can be considered a smaller processor architecture that supports instruction parallelism (multiple issues). Streaming multiprocessors are the execution platform for thread blocks, but generally do not support out-of-order execution. Each individual warp on a streaming multiprocessor executes the same instruction in a Single Instruction Multiple Data (SIMD) manner.

[0030] In addition, to facilitate understanding of this solution, the application background of the embodiments of this application will be explained before describing the embodiments of this application.

[0031] As a traditional parallel computing processor chip, the GPU was originally used for fixed-pipeline image rendering. With the increasing demand for general computing, its computing core has become more like a general-purpose processor. In addition, its internal ALU resources, storage bandwidth and other design features enable the GPU to perform a certain amount of parallel data computing, thus providing powerful computing support.

[0032] In related technologies, a matrix multiplier ALU resource is typically added to a traditional GPU architecture. This reuses the memory bandwidth and general-purpose computing unit parallelism of the original GPU, providing input data movement and additional auxiliary computation for matrix operations. However, if the size of the matrix multiplier continues to increase, when dealing with larger matrices, it is usually limited by the existing GPU's hierarchical structure, memory bandwidth, and the size of general-purpose computing units. Consequently, some input matrix operations cannot be efficiently moved to the matrix multiplier's input buffer. Therefore, even if the matrix multiplier's ALU resource exceeds a certain threshold, it is easily limited by the GPU's memory bandwidth, making it difficult to efficiently utilize the ALU resources of the matrix multiplier within the GPU.

[0033] To address the aforementioned issues, this application provides a resource allocation method, apparatus, and medium. By employing mutex locks, it enables N sequentially running threads within the same thread group and thread queue of a streaming multiprocessor with N thread blocks to exclusively occupy the hardware computing resources associated with the N thread blocks for a preset time period. In other words, it concatenates the matrix operation units associated with each of the N thread blocks, synchronously processing larger matrices. By synchronizing the N sequentially running threads within the same thread group and thread queue to share the input data of the hardware computing resources associated with the N thread blocks, it synchronizes the computing resources within the streaming multiprocessor for computation and data transfer, thereby enabling the streaming multiprocessor to provide stronger computing power.

[0034] This application provides a resource allocation method applied to a streaming multiprocessor running N thread blocks, as described in the following embodiments. Figure 1 As shown, the resource allocation method includes steps S101 to S103, wherein:

[0035] Step S101: In response to receiving a set of thread queue mutex requests, based on the thread identifier corresponding to the set of thread queue mutex requests, the request information in at least one thread queue mutex tracker that matches the set of thread queue mutex requests is adjusted using a mutex lock mechanism to obtain at least one adjusted thread queue mutex tracker.

[0036] The thread queue mutual exclusion request set includes: mutual exclusion requests from M threads belonging to different thread queues sent by each thread block. Each thread queue consists of N sequentially connected threads, and the N threads correspond to N thread blocks. M and N are both integers greater than 0.

[0037] In some embodiments of this application, the mutex mechanism is implemented by hardware circuitry, which includes a thread queue mutex tracker (including a request mask register and an allocation mask register) for storing the request status and resource allocation status of the thread queue.

[0038] In some embodiments of this application, N thread blocks run in a streaming multiprocessor (SM), and the value of N can be determined as needed; for example, N is 1, 2, 3, etc.; 4 thread blocks run in one SM.

[0039] The front-end module corresponding to SM can break down the compute kernel into several thread groups and send some of the thread groups into SM for execution. Here, the thread groups sent into SM can be further broken down into several work items. At the same time, several work items can be packaged into threads (tasks) and sent into SM for execution after completing various necessary resource requests.

[0040] Here, the same thread group will only run in the same SM. The thread group can be broken down into K (N integer multiples of natural numbers) more threads with consecutive time, and every four consecutive threads can form a thread team.

[0041] In some embodiments of this application, a mutex management module in the SM can be used to receive thread queue mutual exclusion requests from M threads sent by each thread block. That is, the set of thread queue mutual exclusion requests includes N (thread blocks) × M thread queue mutual exclusion requests. Here, the mutex management module interacts with the N thread blocks to receive M thread queue mutual exclusion requests sent by each of the N thread blocks; wherein the N threads corresponding to the N × M thread queue mutual exclusion requests can be in multiple different thread queues. Further, each of the N thread blocks in the same SM can simultaneously send M thread queue mutual exclusion requests to the mutex management module according to the order of the thread's corresponding identifier size.

[0042] For example, the SM internally operates with: block0, block1, block2, and block3 (here, "block" represents a thread block), and the SM internally contains: thread queue 1 consisting of sequentially occurring threads 0 to 3, thread queue 2 consisting of sequentially occurring threads 4 to 7, thread queue 3 consisting of sequentially occurring threads 8 to 11, and thread queue 4 consisting of sequentially occurring threads 12 to 15; here, block0 sends thread queue mutex request 0 for thread 0, thread queue mutex request 1 for thread 5, and thread queue mutex request 2 for thread 11; block1 sends thread queue mutex request 3 for thread 9, thread queue mutex request 4 for thread 14, and thread queue mutex request 5 for thread 6; block2 sends thread queue mutex request 6 for thread 7, thread queue mutex request 7 for thread 2, and thread queue mutex request 8 for thread 12; block3 sends thread queue mutex request 9 for thread 3, thread queue mutex request 10 for thread 5, and thread queue mutex request 11 for thread 10.

[0043] In some embodiments of this application, the N×M thread queue mutex request set includes not necessarily all valid data; that is, any thread queue mutex request sent by any thread block can be a valid request or an invalid request.

[0044] It should be noted that the thread queue mutex request can be sent by a thread when it is running the instruction corresponding to the thread queue mutex request, based on the corresponding thread block.

[0045] It should be noted that the value of M can be determined by the hardware architecture corresponding to the thread block in the streaming multiprocessor; for example, M can be 1, 2, etc.

[0046] In some embodiments of this application, the thread queue mutex tracker may be a register or a group of registers, which may be used to store relevant information and record the status information corresponding to the relevant thread queue mutex requests.

[0047] In some embodiments of this application, the number of thread queue mutex trackers can be one, two or more; when the number is two or more, the data stored inside the different thread queue mutex trackers is different.

[0048] In some embodiments of this application, the state corresponding to the request information in each thread queue mutex tracker can be identified first. Based on the state and N×M thread queue mutex requests, the request information in each thread queue mutex tracker can be adjusted according to certain writing rules (e.g., mutex lock mechanism) to obtain at least one thread queue mutex tracker after adjustment.

[0049] Alternatively, a mutex lock mechanism can be used to adjust the request information in at least one thread queue mutex tracker based on the thread identifier corresponding to the thread queue mutex request set (here, since each thread queue mutex request corresponds to one thread, each thread queue mutex request has its own corresponding thread identifier), so as to obtain at least one thread queue mutex tracker after adjustment.

[0050] In some embodiments of this application, the request information of some thread queue mutex trackers in at least one thread queue mutex tracker can be adjusted based on the thread queue mutex request set, or the request information in each thread queue mutex tracker in at least one thread queue mutex tracker can be adjusted to obtain at least one thread queue mutex tracker with adjustment.

[0051] It should be noted that when there are two or more mutex trackers for at least one thread queue, the request information in the mutex trackers for different thread queues will be different.

[0052] In some embodiments of this application, the occupancy status of the thread queue mutex tracker can be represented by req_mask, which can be used to record whether the request data of the thread queue mutex tracker is empty or valid. For example, when req_mask[i]=0, it indicates that the occupancy status of the thread queue mutex tracker is unoccupied, that is, there is no request information stored in the thread queue mutex tracker (i.e., the request information is empty). When req_mask[i]=1, it indicates that the occupancy status of the thread queue mutex tracker is occupied, that is, the thread queue mutex tracker stores request information (i.e., it stores valid request information). Here, if it is valid, more specific details can be determined based on [i].

[0053] Step S102: Among the adjusted at least one thread queue mutex tracker, determine the target tracker that meets the resource allocation conditions.

[0054] In some embodiments of this application, any target tracker that meets the relevant conditions can be selected from at least one adjusted thread queue mutex tracker; here, the resource allocation condition can be that both the occupancy state and the allocation state of the target tracker meet the resource allocation condition; for example, the occupancy state can include occupied or unoccupied, and the allocation state can include unallocated or allocated.

[0055] Here, the number of target trackers can be one, two, or more.

[0056] In some embodiments of this application, if the resource allocation condition is that the thread queue mutex tracker is occupied and the allocation status is unallocated, then the above step S102 can be implemented through the following process:

[0057] In at least one adjusted thread queue mutex tracker, determine the target tracker whose corresponding occupancy status is occupied and whose corresponding allocation status is unallocated.

[0058] Here, any two thread queue mutexes in at least one thread queue mutex tracker may have the same or different occupancy and allocation states respectively.

[0059] The number of target trackers can be one, two, or more.

[0060] It should be noted that this occupancy status can be used to describe whether the thread queue mutex tracker stores request information. If it does, the corresponding occupancy status is occupied (i.e., the thread queue mutex tracker is occupied by request information). If it does not store, i.e., the request information is empty, the corresponding occupancy status is unoccupied.

[0061] Correspondingly, this allocation status can be used to describe whether the thread has been allocated corresponding computing resources for its requests within the mutex tracker, and can thus be divided into two categories: allocated and unallocated.

[0062] In this way, the relevant states of at least one adjusted thread queue mutex tracker are retrieved to find the target tracker that meets the resource allocation conditions. This provides a foundation for subsequently using the matrix operation units associated with each of the N sequentially occurring threads in the same thread queue within the same thread group, thereby enabling the concatenation of the matrix operation units associated with each of the N thread blocks to provide greater computing power support.

[0063] Step S103: Based on the allocation request corresponding to the target thread identifier in the target tracker, allocate the computing resources associated with N thread blocks to the thread queue associated with the target thread identifier.

[0064] In some embodiments, based on the allocation request corresponding to the target thread identifier in the target tracker, the computing resources associated with N thread blocks can be concatenated into a target computing resource and allocated to the thread queue associated with the target thread identifier, so that the N threads in the target thread queue can use the target computing resource synchronously.

[0065] In some embodiments, the computing resources are combinable hardware resources; for example, the hardware resources can be matrix operation units. The above-described allocation of computing resources associated with N thread blocks to the thread queue associated with the target thread identifier can be understood as: concatenating the matrix operation units associated with N thread blocks into a target matrix operation unit, and allocating the target matrix operation unit to the thread queue associated with the target thread identifier; the N threads in the target thread queue synchronously use the target computing resources to perform matrix operations.

[0066] In some embodiments of this application, based on the allocation request corresponding to the target thread identifier, the computing resources associated with N thread blocks within the SM, such as the matrix operation hardware resources (multipliers, adders, etc. required for matrix operations) corresponding to each thread block, can be allocated only to the thread queue associated with the target thread identifier within a preset time period. Here, the thread queue associated with the target thread identifier also includes N sequentially consecutive target threads. Therefore, the N computing resources associated with the N thread blocks can be allocated only to the N sequentially consecutive target threads within the preset time period.

[0067] It should be noted that because the N sequentially occurring threads within the thread queue associated with the target thread identifier may run (execute or issue) to the thread queue mutex request in different sequences (i.e., the N×M thread queue mutex requests received at the same time may not necessarily include the thread queue mutex requests corresponding to the N sequentially occurring threads that make up the target thread queue), when the allocation request corresponding to the target thread identifier takes effect, the computing resources associated with the N thread blocks begin to wait for allocation to the thread queue associated with the target thread identifier. In the following period, the computing resources associated with the N thread blocks will only be used by the N sequentially occurring threads within the thread queue associated with the target thread identifier until all N sequentially occurring threads within the thread queue associated with the target thread identifier have run to the mutex release request, and then the N threads within the thread queue associated with the target thread identifier will be released from the computing resources associated with the N thread blocks.

[0068] In this way, within the time period from receiving a mutex request from a relevant thread to its corresponding mutex release request, the computational resources associated with N thread blocks—that is, matrix operation units—can synchronously perform some matrix operations (for use by N sequentially connected threads within the thread queue of the relevant thread). Here, due to the uniqueness of the matrix operation unit, N sequentially connected tasks in different thread queues need to reuse the matrix operation unit separately in time. The mutex mechanism enables efficient utilization of the computational resources associated with the N thread blocks.

[0069] It should be noted that since each thread block sends a mutual exclusion request for M threads in different thread queues, and the stream multiprocessor consists of N thread blocks, each thread queue comprises N sequentially occurring threads, the N sequentially occurring threads in each thread queue run in N thread blocks respectively. For example, when N is 4, threads 0 to 3 in the same thread queue run sequentially in thread blocks 0 to 3.

[0070] Thus, through the resource allocation method described above, by adjusting the thread queue mutex tracker based on the thread queue mutex request set using a mutex lock mechanism, N sequentially occurring threads in the same thread group and the same thread queue can ensure their exclusive access to relevant resources during the resource allocation process through the mutex lock mechanism. Specifically, after determining the target tracker that meets the resource allocation conditions, the computing resources associated with the N thread blocks are allocated to the thread queue associated with the target thread identifier. This enables these N sequentially occurring threads running within the streaming multiprocessor to exclusively occupy the hardware computing resources associated with the N thread blocks. Furthermore, when these N sequentially occurring threads exclusively occupy the hardware computing resources (taking matrix operation units as an example), it is equivalent to concatenating the matrix operation units associated with each of the N thread blocks. Because these threads are sequentially occurring and in the same thread group and the same thread queue, they can perform synchronous computation, thereby enabling the processing of larger matrices. At the same time, these threads running synchronously can share the input data of the hardware computing resources associated with the N thread blocks. This synchronous operation and data sharing mechanism enables the computing resources within the streaming multiprocessor to be synchronized for computation and data transfer. When processing partial input matrix operations, this synchronization mechanism allows data to be moved more efficiently to the input buffer of the matrix multiplier, thereby providing stronger computing power support for the streaming multiprocessor.

[0071] In some embodiments of this application, at least one thread queue mutex tracker can be determined first, and then the request information within the at least one thread queue mutex tracker can be adjusted. That is, step S101 provided in the above embodiments can be implemented by steps S201 and S202, such as... Figure 2 As shown, where:

[0072] Step S201: In response to receiving a set of thread queue mutex requests, determine at least one thread queue mutex tracker.

[0073] In some embodiments of this application, after receiving a set of thread queue mutex requests, at least one thread queue mutex tracker corresponding to each thread queue mutex request can be retrieved based on the information carried by each thread queue mutex request in the set.

[0074] In some embodiments of this application, the N×M thread queue mutex requests in the received thread queue mutex request set can be reordered, and based on this reordering and the information carried by each thread queue mutex request, the corresponding thread queue mutex tracker can be searched sequentially to obtain at least one thread queue mutex tracker. That is, step S201 provided in the above embodiments can be implemented by the following steps S2011 and S2012 (not shown in the figure):

[0075] Step S2011: In response to receiving the thread queue mutex request set, determine the index number corresponding to each thread queue mutex request in the thread queue mutex request set.

[0076] In some embodiments of this application, in response to receiving N×M thread queue mutual exclusion requests (thread queue mutual exclusion request set), the received N×M thread queue mutual exclusion requests can be reordered, and the index number corresponding to each thread queue mutual exclusion request can be determined based on the reordering; here, the reordering can be sorted according to a predetermined sorting rule.

[0077] The index number can be represented by numbers or text; here, the index number can be used to represent the mutual exclusion requests of each thread team, and the order of the mutual exclusion requests of N×M thread teams.

[0078] In some embodiments of this application, the index number corresponding to each thread queue mutual exclusion request can be generated by the different thread blocks corresponding to each thread queue mutual exclusion request in N×M thread queue mutual exclusion requests and the different threads corresponding to different thread blocks. That is, step S2011 provided in the above embodiments can be implemented by the following process (not shown in the figure):

[0079] The first step is to determine the first identifier of each thread queue mutex request in response to the received set of thread queue mutex requests; the first identifier is the unique identifier of the thread block corresponding to the thread queue mutex request.

[0080] In some embodiments of this application, the first identifier is a unique identifier of the thread block corresponding to the thread queue mutual exclusion request, used to distinguish different thread blocks. For example, if 4 (N is 4) thread blocks are running in a streaming multiprocessor, then the first identifier of the thread block can be represented by thread block 0 to thread block 3, or by block0 to block3.

[0081] For example, the first identifier of the thread block corresponding to thread queue mutex request 1 is: block0; the first identifier of the thread block corresponding to thread queue mutex request 2 is: block3; the first identifier of the thread block corresponding to thread queue mutex request 3 is: block0; the first identifier of the thread block corresponding to thread queue mutex request 4 is: block2, etc.

[0082] The second step is to determine the second identifier of each thread queue mutex request; the second identifier is the request sequence number of the thread queue mutex request in the request set within the thread block; the request set within the thread block includes the sequence number of the thread requests within the thread block, and the request set within the thread block is all thread queue mutex requests sent by the thread block corresponding to the thread queue mutex request.

[0083] In some embodiments of this application, the second identifier is the request sequence number of the thread queue mutex request within the request set of the thread block. In some possible implementations, the second identifier may indicate the sequential position of the request within its respective thread block, such as an incrementing sequence number assigned according to the request arrival time.

[0084] For example, thread block 1 sends thread queue mutex request 0, thread queue mutex request 1, and thread queue mutex request 2.

[0085] The third step is to generate the index number corresponding to the mutual exclusion request of each thread based on the first and second identifiers.

[0086] The index number is a unique index generated based on the first identifier and the second identifier. It is used to quickly locate or manage mutual exclusion requests for specific thread queues, and can be generated by combining them through a hash function or concatenation method.

[0087] In some embodiments of this application, for each thread queue mutex request, the first identifier and the second identifier of the thread queue mutex request can be concatenated to obtain the index number corresponding to the thread queue mutex request, and then assigned to the thread queue mutex request accordingly.

[0088] For example, each block sends 3 task requests, which are scattered according to the block interleave rule, and are ordered from low to high index number as follows: block0_req0, block1_req0, block2_req0, block3_req0, block0_req1, block1_req1, block2_req1, block3_req1, block0_req2, block1_req2, block2_req2, block3_req2.

[0089] In this way, the mutual exclusion requests for M threads sent by N thread blocks are used to generate an index number for each mutual exclusion request, based on the corresponding thread block and the corresponding thread within that thread block. This provides a foundation for subsequently determining at least one thread queue mutual exclusion tracker based on the sorting corresponding to the index number and the relevant information carried in the mutual exclusion requests.

[0090] Step S2012: According to the sorting corresponding to the index number, determine the thread queue mutex tracker corresponding to each thread queue mutex request based on the thread queue mutex identifier carried by each thread queue mutex request, and obtain at least one thread queue mutex tracker.

[0091] In some embodiments of this application, the thread queue mutex request set consists of N×M thread queue mutex requests, which are sorted according to their corresponding index numbers, and the matching thread queue mutex tracker is retrieved sequentially based on the thread queue mutex identifier carried inside each request.

[0092] It's important to note that all threads within the same thread group use the same code segment. Therefore, since the code segments are identical, the tasks they perform are essentially the same, and the hardware resources used for these tasks are also identical. Consequently, the thread queue mutex requests from all threads within the same thread group carry the same mutex identifier. However, since a thread group can include multiple different thread queues, it's possible for multiple threads in different thread queues but within the same thread group to carry the same mutex identifier in their mutex requests.

[0093] In this way, based on the index number corresponding to each thread queue mutex request and the thread queue mutex identifier carried by each thread queue mutex request, the thread queue mutex tracker corresponding to each thread queue mutex request is determined, resulting in at least one thread queue mutex tracker. Thus, by sequentially searching for the thread queue mutex tracker corresponding to each thread queue mutex request in a certain order, at least one thread queue mutex tracker can be obtained more accurately and comprehensively.

[0094] Step S202: Based on the thread identifier corresponding to the thread queue mutex request set, the request information in at least one thread queue mutex tracker is adjusted using the mutex lock mechanism to obtain at least one adjusted thread queue mutex tracker.

[0095] One thread queue mutex tracker corresponds to one thread queue, and the request information in the thread queue mutex tracker is used to record the request status of the corresponding thread queue.

[0096] It is understandable that when adjusting request information using the mutex mechanism, in this application, one thread queue mutex tracker corresponds to only one thread queue. That is, one thread queue mutex tracker only processes requests belonging to the corresponding thread queue, thereby achieving mutual exclusion of resource allocation.

[0097] In some embodiments of this application, the request information in at least one thread queue mutex tracker can be adjusted based on the thread identifiers carried in the N×M thread queue mutex request set and using a mutex lock mechanism to obtain at least one adjusted thread queue mutex tracker.

[0098] Here, the occupancy status of the thread queue mutex tracker can be identified first. Then, based on the occupancy status and related thread identifiers, and using the mutex lock mechanism, the request information in each thread queue mutex tracker can be adjusted to obtain the adjusted thread queue mutex trackers.

[0099] In this way, based on the thread identifiers corresponding to the mutual exclusion request sets of thread queues, the request information in the mutual exclusion trackers of each thread queue is adjusted using the mutual exclusion lock mechanism, resulting in adjusted mutual exclusion trackers for each thread queue. This provides a foundation for ensuring that the computing resources associated with each subsequent thread block have an equal chance of being allocated to N sequentially connected threads in the same thread queue.

[0100] Following the description above, based on the occupancy status of each thread queue mutex tracker (whether the request information is empty), the thread identifiers carried in the N×M thread queue mutex requests can be used to adjust the request information of each thread queue mutex tracker, resulting in adjusted thread queue mutex trackers. That is, step S202 can be implemented based on steps S301 and S302, such as... Figure 3 As shown, where:

[0101] Step S301: Determine the occupancy status of the mutex tracker for each thread.

[0102] In some embodiments of this application, the occupancy status of each thread queue mutex tracker can be represented by unoccupied or occupied states.

[0103] The occupancy status of the mutex tracker for different threads can be the same or different.

[0104] Step S302: Using the mutex lock mechanism, based on the occupancy status and the thread identifier corresponding to the thread queue mutex request set, adjust the request information in each thread queue mutex tracker to obtain the adjusted thread queue mutex tracker.

[0105] In some embodiments of this application, based on the mutex lock mechanism, the request information in each thread queue mutex tracker can be adjusted (written, changed, kept unchanged, etc.) based on the occupancy state and N×M thread queue mutex requests to obtain the adjusted thread queue mutex tracker.

[0106] Based on the embodiments disclosed in this application, by determining the occupancy status of each thread queue mutex tracker, it is possible to clearly understand whether each tracker is currently in use, providing basic information for subsequent resource allocation and request processing, and helping to rationally plan the use of resources; at the same time, by using the mutex lock mechanism, the request information in each thread queue mutex tracker can be adjusted based on the occupancy status and the thread identifier corresponding to the thread queue mutex request set, and the request information can be updated accurately and orderly according to the thread identifier and the tracker occupancy status.

[0107] In some embodiments of this application, based on the different occupancy states of the thread queue mutex trackers, the following two methods can be further implemented to obtain adjusted thread queue mutex trackers. That is, step S302 provided in the above embodiments can be implemented in the following two ways:

[0108] Method 1 includes the following steps S3021 and S3022 (not shown in the figure):

[0109] Step S3021: In at least one thread queue mutex tracker, if the corresponding first thread queue mutex tracker is not occupied, determine the first thread queue mutex request carrying the first thread queue mutex identifier in the thread queue mutex request set.

[0110] Among them, the mutex identifier of the first thread queue is the mutex identifier corresponding to the mutex tracker of the first thread queue.

[0111] In some embodiments of this application, the number of the first thread queue mutex trackers can be one, two or more; here, we take one first thread queue mutex tracker as an example.

[0112] Here, the first thread queue mutex identifier is the information carried by the first thread queue mutex request and the first thread queue mutex tracker; that is, as mentioned above, based on the thread queue mutex identifier carried inside each thread queue mutex request, the thread queue mutex tracker with the same thread queue mutex identifier is retrieved.

[0113] It should be noted that the mutex flag for the first thread queue can be described using any field.

[0114] In some embodiments of this application, the occupancy status of the first thread queue mutex tracker is unoccupied, which means that the request information in the first thread queue mutex tracker is empty.

[0115] Step S3022: The thread identifier corresponding to the mutex request of the first thread queue is used as the request information in the mutex tracker of the first thread queue to obtain the adjusted mutex tracker of the first thread queue.

[0116] In some embodiments of this application, the thread identifier corresponding to the first thread queue mutex request, i.e., the identifier information corresponding to the thread, can be used as the request information of the first thread queue mutex tracker. That is, the thread identifier is written into the first thread queue mutex tracker to obtain the adjusted first thread queue mutex tracker.

[0117] In this way, for the mutex tracker of the first thread queue that is not occupied, the thread identifier corresponding to the mutex request of the first thread queue that matches it can be used as the request information in the mutex tracker of the first thread queue, thereby providing a basis for allocating relevant computing resources to the N sequentially consecutive threads in the relevant thread queue.

[0118] It should be noted that if there are N×M thread queue mutex requests in the thread queue mutex request set, and there are multiple thread queue mutex requests corresponding to multiple threads in the same thread group, the thread queue mutex requests corresponding to multiple threads in the same thread group will carry the same thread queue mutex identifier, and multiple threads belonging to the same thread group do not necessarily belong to the same thread queue.

[0119] Here, when the first thread queue mutex request includes at least two candidate thread queue mutex requests, that is, in the set of thread queue mutex requests, the number of thread queue mutex requests that match the first thread queue mutex tracker is at least two; adaptive selection can be made from at least two candidate thread queue mutex requests, that is, step S3022 provided in the above embodiment can be implemented by the following process:

[0120] The first step is to select the middle thread queue mutex request that is ranked first among at least two candidate thread queue mutex requests.

[0121] The second step is to use the thread identifier corresponding to the mutex request of the intermediate thread queue as the request information in the mutex tracker of the first thread queue, so as to obtain the adjusted mutex tracker of the first thread queue.

[0122] In some embodiments of this application, among at least two candidate thread queue mutual exclusion requests, the middle thread queue mutual exclusion request with the higher index number can be selected based on the index number corresponding to each of the at least two candidate thread queue mutual exclusion requests.

[0123] In this process, the thread identifier corresponding to the intermediate thread queue mutex request can be used to identify which thread group and which thread queue the thread belongs to. This thread identifier is then used as the request information for the first thread queue mutex tracker, i.e., written into the first thread queue mutex tracker to obtain the adjusted first thread queue mutex tracker.

[0124] In this way, for a first-thread-team mutex tracker that matches multiple thread-team mutex requests, the index numbers corresponding to the matched multiple thread-team mutex requests can be used as a filtering condition, so that the first-thread-team mutex tracker only stores the thread identifiers corresponding to one type of request. Thus, based on a relatively convenient execution logic, it provides a foundation for ensuring that the computing resources associated with each thread block have an equal chance of being allocated to N sequentially connected threads in the same thread team.

[0125] Here, after executing step S3022 and the second step described above, that is, after the corresponding request information is written into the first thread queue mutex tracker, the occupancy status of the adjusted first thread queue mutex tracker can be changed from unoccupied to occupied. That is, the resource allocation method provided in the above embodiment can also execute the following step A:

[0126] Step A: In response to obtaining the adjusted first thread queue mutex tracker, change the occupancy status of the adjusted first thread queue mutex tracker from unoccupied to occupied.

[0127] In some embodiments of this application, the occupancy status of the adjusted first thread queue mutex tracker is changed from unoccupied to occupied based on the writing of request information; for example, the request information in the first thread queue mutex tracker is: req_mask[i]=0, and correspondingly, the request information in the adjusted first thread queue mutex tracker is: req_mask[i]=1; where i can represent the thread identifier corresponding to the first thread queue mutex request, or the thread identifier corresponding to the intermediate thread queue mutex request.

[0128] In this way, based on the writing of request information, the occupancy status of the first thread queue mutex tracker is changed from unoccupied to occupied, so as to perform data management on at least one thread queue mutex tracker, and to provide a basis for the subsequent allocation of hardware computing resources associated with N thread blocks to four sequentially connected threads through mutex locks.

[0129] Method 2 includes the following steps S3023 to S3025 (not shown in the figure):

[0130] Step S3023: If, in at least one thread queue mutex tracker, there exists a second thread queue mutex tracker whose occupancy status is occupied, determine the second thread queue mutex request carrying the second thread queue mutex identifier in the thread queue mutex request set.

[0131] The mutex identifier for the second thread queue is the mutex identifier corresponding to the mutex tracker for the second thread queue.

[0132] In some embodiments of this application, the number of the second thread queue mutex trackers can be one, two or more; here, we take one second thread queue mutex tracker as an example.

[0133] Here, the second thread queue mutex tracker can also refer to the modified first thread queue mutex tracker mentioned above.

[0134] The description of the mutex flag for the second thread queue can be found in the description of the mutex flag for the first thread queue mentioned above, and will not be repeated here.

[0135] In some embodiments of this application, the second thread queue mutex tracker is in an unoccupied state, which indicates that the first thread queue mutex tracker stores request information.

[0136] Step S3024: Determine the consistency judgment result based on the thread identifier corresponding to the mutex request of the second thread queue and the current identifier corresponding to the request information in the mutex tracker of the second thread queue.

[0137] The consistency determination result is used to characterize whether the mutex request of the second thread team and the request in the mutex tracker of the second thread team are in the same thread team of the same thread group.

[0138] In some embodiments, the thread identifier corresponding to the mutex request of the second thread queue can be extracted and compared with the current identifier recorded in the mutex tracker of the second thread queue. If the two are the same, they are determined to be consistent; otherwise, they are determined to be inconsistent.

[0139] Step S3025: Based on the consistency determination result, adjust the request information in the second thread queue mutex tracker to obtain the adjusted second thread queue mutex tracker.

[0140] In some embodiments of this application, the current identifier corresponding to the request information in the second thread queue mutex tracker can be identified first, and then the request information in the second thread queue mutex tracker can be adjusted based on the thread identifier corresponding to the second thread queue mutex request and the current identifier, thereby obtaining the adjusted second thread queue mutex tracker.

[0141] In this way, for the mutex tracker of the second thread queue whose occupancy status is occupied, based on the common consideration of the current identifier stored in its internal memory and the thread identifier corresponding to the mutex request of the second thread queue, the relevant computing resources can be allocated to N sequentially consecutive threads in the same thread queue through the mutex mechanism.

[0142] In some embodiments, the request information in the second thread queue mutex tracker can be reasonably adjusted according to the consistency determination result to adapt to the processing requirements of thread queue mutex requests under different circumstances. If the two are consistent, the thread identifier corresponding to the second thread queue mutex request is also written into the second thread queue mutex tracker to obtain the adjusted second thread queue mutex tracker; if the two are inconsistent, the second thread queue mutex tracker can remain unchanged. That is, step S3025 provided in the above embodiment can be implemented in the following two ways:

[0143] In scenario one, if the consistency determination result indicates that the second thread queue mutex request and the request in the second thread queue mutex tracker are in the same thread group and the same thread queue, the thread identifier and the current identifier corresponding to the second thread queue mutex request are used together as the request information in the second thread queue mutex tracker to obtain the adjusted second thread queue mutex tracker.

[0144] In the second scenario, if the consistency determination result indicates that the mutual exclusion request of the second thread team and the request in the second thread team mutual exclusion tracker are not in the same thread team of the same thread group, the current identifier corresponding to the request information in the second thread team mutual exclusion tracker remains unchanged, and the second thread team mutual exclusion tracker is obtained.

[0145] In one embodiment of this application, the thread identifier corresponding to the second thread queue mutex request can be matched with the current identifier. If the two match, the thread identifier corresponding to the second thread queue mutex request is also written into the second thread queue mutex tracker to obtain the adjusted second thread queue mutex tracker. If the two do not match, the second thread queue mutex tracker can remain unchanged.

[0146] It is understandable that the purpose of the above "jointly serving as request information within the second thread queue mutex tracker" is to complete the N thread identifiers of the thread queue, that is, to add the thread identifier that matches the current identifier to the request information until the request information contains all N thread identifiers of the thread queue. This way, they can correspond to N thread blocks respectively, so as to meet the requirements of resource splicing for a complete thread queue.

[0147] It should be noted that, based on the writing rules described above, the thread identifiers corresponding to the request information in the second thread queue mutex tracker can include the identifiers corresponding to N sequentially consecutive threads that are in the same thread group and the same thread queue. In other words, the request information in any thread queue mutex tracker can store up to N thread identifiers, which are the identifiers corresponding to threads that are in the same thread group and the same thread queue.

[0148] In this way, for the mutex tracker of the second thread queue that is in an occupied state, it can be matched with its internally stored current identifier and the thread identifier corresponding to the mutex request of the second thread queue. If they match, they are used together as the request information; if they do not match, the mutex tracker of the second thread queue remains unchanged. This provides a basis for allocating relevant computing resources to N sequentially connected threads within the relevant thread queue.

[0149] In this way, based on the occupancy status of each thread queue's mutex tracker (whether the request information is empty), the thread identifiers carried in the N×M thread queue mutex requests can be used to adjust the request information of each thread queue's mutex tracker, resulting in adjusted mutex trackers. This provides a foundation for subsequently concatenating the matrix operation units associated with each of the N thread blocks to synchronously process larger matrices. This enables the synchronous execution of N sequentially consecutive threads in the same thread group and thread queue, sharing the input data of the hardware computing resources associated with the N thread blocks, thereby allowing the streaming multiprocessor to provide stronger computing power support.

[0150] In some embodiments of this application, based on the allocation request corresponding to the target thread identifier within the target tracker, N computing resources associated with N thread blocks can be provided to N time-sequentially consecutive target threads. That is, step S103 provided in the above embodiments can be implemented using steps S401 and S402, such as... Figure 4 As shown, where:

[0151] Step S401: Based on the allocation request, the N computing resources associated with the N thread blocks are concatenated to obtain the target computing resource.

[0152] In some embodiments of this application, the target thread identifier corresponding to the request information inside the target tracker (i.e., the written request information mentioned above, the thread identifier corresponding to the mutual exclusion request of the relevant thread team: the thread identifier corresponding to the mutual exclusion request of the first thread team, the thread identifier corresponding to the mutual exclusion request of the second thread team) can be identified first, and then the allocation request corresponding to the target thread identifier can be generated.

[0153] Here, the allocation request can be sent to the relevant processing module within the streaming multiprocessor to request the use of N computing resources associated with N thread blocks within a preset time period, and then combine them into a whole to form the target running resource.

[0154] Step S402: Allocate the target computing resources to N time-sequential target threads for synchronous use.

[0155] The target thread identifier is associated with a thread team consisting of N sequentially connected target threads.

[0156] In some embodiments of this application, the target computing resources can be allocated to N time-sequential target threads. This can be achieved by allocating each of the N computing resources to one of the N time-sequential target threads.

[0157] It's important to note that the times when N threads in the same thread group and thread queue reach the thread queue mutex request are different. That is, the time it takes for a thread queue mutex tracker to receive these requests is not uniform. Therefore, it takes some time for the tracker to receive these requests. If the allocation request for any one of the N threads in the same thread group and thread queue is executed, then the N computing resources associated with the N thread blocks can only be allocated to the thread queue corresponding to that thread for a given period of time.

[0158] This allows for the concatenation of matrix operation units within N thread blocks, enabling synchronous processing of larger matrices (N sequentially connected threads) and sharing of input data for matrix operations. Consequently, it enables the synchronization of resources within existing computing cores for computation and data transfer, thereby improving computing power.

[0159] Here, after the allocation request for the target tracker is generated and executed, the allocation status of the target tracker can be changed from unallocated to allocated. That is, the resource allocation method provided in the above embodiment can also perform the following step B:

[0160] Step B changes the target tracker's allocation status from unallocated to allocated.

[0161] In this case, the computing resources corresponding to the target tracker in the allocated state are unavailable to other thread teams.

[0162] It is understandable that the computing resources corresponding to the target tracker in the allocated state are only used by the current thread team and are not available to other thread teams, thus implementing a mutual exclusion restriction on other thread teams.

[0163] In some embodiments of this application, after allocating the computing resources associated with N thread blocks to the thread queue associated with the target thread identifier, the allocation status of the target tracker can be changed from unallocated to allocated.

[0164] In this way, the allocation state of the target tracker is adaptively changed based on the execution of the allocation request, so as to manage the data of the target tracker.

[0165] In some embodiments of this application, the mutual exclusion release requests from each target thread can be further received, and the allocation of related computing resources and related information can be stopped accordingly. That is, the resource allocation method provided in the above embodiments can also perform the following steps C1 and C2:

[0166] Step C1: In response to receiving mutual exclusion release requests from each target thread, stop allocating target computing resources to N target threads.

[0167] Step C2: Reset the allocation status of the target tracker to unallocated and reset the occupancy status of the target tracker to unoccupied.

[0168] In some embodiments of this application, the allocation of target computing resources to the N target threads can be stopped only after receiving a mutex release request from each of the N sequentially occurring target threads. For example, if the N target threads include threads 1 to 4, then the allocation of target computing resources to the N target threads can be stopped only after receiving a mutex release request from each of the threads 1 to 4.

[0169] After releasing N target threads from the target computing resources, the allocation and occupancy states of the target tracker can be reset respectively.

[0170] In this way, based on the received mutex release request, the corresponding resource release and related state reset are executed. This enables efficient and convenient management of the computing resources and related trackers associated with N thread blocks, thus providing a foundation for stronger computing power support for streaming multiprocessors.

[0171] The resource allocation method described above will be explained below with reference to a specific embodiment. However, it is worth noting that this specific embodiment is only for better illustrating the embodiments of this application and does not constitute an improper limitation on the embodiments of this application.

[0172] GPUs, as traditional parallel computing processor chips (compared to central processing units (CPUs), have more computing cores, typically ranging from hundreds to thousands; the cores of a GPU process multiple tasks simultaneously through parallel computing), were initially used for fixed-pipeline image rendering. As the demand for general-purpose computing increased, their computing cores tended to be designed like general-purpose processors. Combined with their internal ALU resources, storage bandwidth, and other design features, GPUs are able to perform a certain amount of parallel data computation, thus providing powerful computing support.

[0173] In related technologies, with the widespread application of artificial intelligence, matrix operations have become an increasingly common and urgent need. Typically, an ALU (Automatic Luminary Unit) for matrix multiplication is added to a traditional GPU architecture, reusing the existing GPU's memory bandwidth and the parallelism of general-purpose computing units to provide input data movement and some additional auxiliary computations for matrix operations. However, if the size of the matrix multiplier continues to increase, when dealing with larger matrices, it is usually limited by the existing GPU's hierarchical structure, memory bandwidth, and the size of general-purpose computing units. Consequently, some input matrices cannot be efficiently moved to the matrix multiplier's input buffer. Therefore, even if the ALU resources of the matrix multiplier exceed a certain threshold, they are easily limited by the GPU's memory bandwidth, making it difficult to efficiently utilize the ALU resources of the matrix multiplier within the GPU.

[0174] If we focus on its matrix operation capabilities, considering the requirements of pipelined operations, it is necessary to use large-scale hardware resources, physically or logically combining all the hardware resources within the GPU, which will inevitably be detrimental to graphics processing efficiency.

[0175] Based on this, the resource allocation method provided in this application mainly adds synchronous mutex lock logic to the original GPU architecture without reducing its original graphics rendering capabilities. By combining the current hardware resources in the GPU and performing some operations synchronously, the method enables the matrix multiplier to perform some general auxiliary calculations or move data, thereby enabling the GPU to process larger-scale matrix operations more efficiently.

[0176] like Figure 5 As shown, an SM has 4 thread blocks (blocks) 501; the SM's front-end module will split the compute kernel into several thread groups (workgroups); a certain workgroup will only be sent to one SM, the workgroup will be further split into several workitems, and several workitems will be packed into threads (tasks), and after completing all necessary resource requests, they will be sent to the SM to run (here, the workgroup will be divided into N tasks, and when the compute kernel needs to use a queue mutex, N is a natural number that is a multiple of 4 and less than or equal to 32). Calculated from the order of task allocation, every four consecutive tasks form a thread queue (squad), and the four tasks in each queue run in four blocks inside the SM.

[0177] Here, when a task reaches the `squad mutex req` instruction, it enters the `descheduled` state. Since tasks within the same workgroup and squad reach the `squad mutex req` instruction at different times, and tasks in the `descheduled` state follow a set of logic for being granted squad mutexes according to certain rules, the following applies to mutually exclusive squad mutexes. When several squad tasks (some or all of a squad's tasks) are simultaneously in the `descheduled` state, each squad has an equal probability of being granted a mutex. Once the first task of a squad is granted, that squad mutex is occupied by that squad, and in the following time, it can only grant other tasks from that squad until all four consecutive tasks of that squad have reached the `squad mutex release` instruction. Within the code segment between the `squad mutex req` instruction and the `squad mutex release` instruction, matrix operation units perform some matrix operations synchronously. Due to the uniqueness of matrix operation units, different squares need to reuse matrix operation units separately in time. `squad mutex` ensures mutual exclusion of resource utilization.

[0178] Based on the above description, in the resource allocation method provided in this application embodiment, each block in the same SM will simultaneously send M (here, M is determined based on the hardware architecture corresponding to the block) task square mutex request to the mutex manager 502 in order of task ID size; after the mutex manager receives multiple square mutex request requests:

[0179] First, multiple square mutex request requests will be reordered. Since four consecutive tasks in the same square are allocated to four blocks and are sequential in time, but each square must have an equal probability of being granted, the square mutex request requests of M tasks sent by each block need to be interleave-reordered. This can prevent requests for a certain block from being granted frequently, thus ensuring that each block in the same SM is granted relatively evenly.

[0180] For example, each block sends 3 task requests, which are scattered according to the block interleave rule, and in order of index from low to high: block0_req0, block1_req0, block2_req0, block3_req0, block0_req1, block1_req1, block2_req1, block3_req1, block0_req2, block1_req2, block2_req2, block3_req2.

[0181] Secondly, the mutex manager will perform the following judgments and operations based on the internal queue mutex search and grant logic (5021):

[0182] After the square mutex requests are reordered, the square mutex search logic is responsible for searching each square mutex tracker 0 to N, i.e., the square mutex tracker, and following two rules to determine the TSAK to be written to the relevant square mutex tracker.

[0183] Rule 1: If some square mutex trackers are not occupied, the square mutex search logic can first identify all requests with different square mutex IDs that are currently applying for that square mutex tracker, and then write the information corresponding to all requests with different square mutex IDs into the square mutex tracker.

[0184] Here, if there are multiple requests with different square mutex IDs, the task with the larger index is selected and written into the square mutex tracker.

[0185] Rule 2: If some squad mutex trackers are already in use, a request that matches the workgroup id and squad id stored in that squad mutex tracker can be responded to, and the relevant information will be written into the squad mutex tracker.

[0186] Here, the square mutex search and grant logic can sequentially detect the internal information of each of the 12 requests from low to high index. According to the two rules mentioned above, the task information that meets the two rules is flushed to the found square mutex tracker on the next rising edge of the clock.

[0187] Then, another function of the square mutex search logic is to collect the state in the square mutex tracker and grant the relevant tasks. Grant rules:

[0188] When req_mask[i]=1 and grant_mask[i]=0 in a certain square mutex tracker, use i as the index to obtain the corresponding task id, generate a bit mask as the grant mask, and grant the relevant hardware computing resources to the corresponding task.

[0189] Here, the square mutex tracker internally stores necessary information and records the state of each square mutex. As shown in Table 1 below, the information recorded internally by the square mutex tracker is as follows:

[0190] Table 1

[0191]

[0192] Meanwhile, the Squad mutex release logic receives the squad mutex release signal sent from the block and clears the state of the corresponding squad mutex tracker.

[0193] Here, the tracker with req_mask[i]=1 and grant_mask[i]=0 in each square mutex tracker is detected. The task id of the corresponding block in the task_id_array is found according to i, and the bit mask is generated as the grant_mask to wake up the corresponding task. If a task of a certain block is woken up, the corresponding grant_mask[i] is set to 1 in the next cycle.

[0194] Meanwhile, the queue mutex release module 5023 inside the mutex management module 502, namely the queue mutex releaselogic, receives queue mutex release requests from 4 blocks. The requests include queue_mutex_id information, and it is responsible for resetting req_mask[i] and grant_mask[i] in the corresponding queue_mutex_tracker.

[0195] In this way, by using mutex locks, it enables N sequentially running threads in the same thread group and thread queue within a streaming multiprocessor with N thread blocks to exclusively occupy the hardware computing resources associated with the N thread blocks for a preset time period. In other words, it concatenates the matrix operation units associated with the N thread blocks and synchronously processes larger matrices. By synchronizing the N sequentially running threads in the same thread group and thread queue to share the input data of the hardware computing resources associated with the N thread blocks, it achieves the synchronization of computing resources within the streaming multiprocessor for computation and data transfer, thereby enabling the streaming multiprocessor to provide stronger computing power support.

[0196] This application provides a resource allocation device. Figure 6 This is a schematic diagram of the structural composition of the resource allocation device provided in the embodiments of this application, as shown below. Figure 6 As shown, the resource allocation device 600 is applied to a streaming multiprocessor running N thread blocks. The resource allocation device 600 includes:

[0197] The first processing module 601, in response to receiving a set of thread queue mutex requests, adjusts the request information in at least one thread queue mutex tracker that matches the set of thread queue mutex requests based on the thread identifiers corresponding to the set of thread queue mutex requests and using a mutex lock mechanism, to obtain at least one adjusted thread queue mutex tracker; the set of thread queue mutex requests includes: mutual exclusion requests sent by each thread block to M threads belonging to different thread queues, each thread queue consisting of N sequentially connected threads, and the N threads corresponding to N thread blocks respectively;

[0198] The second processing module 602 is used to determine the target tracker that meets the resource allocation conditions among the adjusted at least one thread queue mutex tracker.

[0199] The third processing module 603 is used to allocate the computing resources associated with N thread blocks to the thread queue associated with the target thread identifier based on the allocation request corresponding to the target thread identifier in the target tracker.

[0200] In some embodiments of this application, the first processing module 601 is further configured to, in response to receiving a set of thread queue mutex requests, determine at least one thread queue mutex tracker; based on the thread identifier corresponding to the set of thread queue mutex requests, adjust the request information in the at least one thread queue mutex tracker using a mutex lock mechanism to obtain an adjusted at least one thread queue mutex tracker; wherein, one thread queue mutex tracker corresponds to one thread queue, and the request information in the thread queue mutex tracker is used to record the request status of the corresponding thread queue.

[0201] In some embodiments of this application, the first processing module 601 is further configured to, in response to receiving a set of thread queue mutual exclusion requests, determine the index number corresponding to each thread queue mutual exclusion request in the set of thread queue mutual exclusion requests; and, according to the sorting corresponding to the index number, determine the thread queue mutual exclusion tracker corresponding to each thread queue mutual exclusion request based on the thread queue mutual exclusion identifier carried by each thread queue mutual exclusion request, thereby obtaining at least one thread queue mutual exclusion tracker.

[0202] In some embodiments of this application, the first processing module 601 is further configured to, in response to receiving a set of thread queue mutual exclusion requests, determine a first identifier for each thread queue mutual exclusion request; the first identifier is a unique identifier of the thread block corresponding to the thread queue mutual exclusion request; determine a second identifier for each thread queue mutual exclusion request; the second identifier is the request sequence number of the thread queue mutual exclusion request in the request set within the thread block; the request set within the thread block includes the sequence number of the thread requests within the thread block, and the request set within the thread block consists of all thread queue mutual exclusion requests sent by the thread block corresponding to the thread queue mutual exclusion request; and generate an index number corresponding to each thread queue mutual exclusion request based on the first identifier and the second identifier.

[0203] In some embodiments of this application, the first processing module 601 is further configured to determine the occupancy status of each thread queue mutex tracker; and, using a mutex lock mechanism, adjust the request information in each thread queue mutex tracker based on the occupancy status and the thread identifier corresponding to the thread queue mutex request set, to obtain the adjusted thread queue mutex tracker.

[0204] In some embodiments of this application, the first processing module 601 is further configured to, when at least one thread queue mutex tracker is in an unoccupied state, determine a first thread queue mutex request carrying a first thread queue mutex identifier in the thread queue mutex request set; wherein, the first thread queue mutex identifier is the mutex identifier corresponding to the first thread queue mutex tracker; and use the thread identifier corresponding to the first thread queue mutex request as the request information in the first thread queue mutex tracker to obtain the adjusted first thread queue mutex tracker.

[0205] In some embodiments of this application, the first thread queue mutex request includes at least two candidate thread queue mutex requests. The first processing module 601 is further configured to select the intermediate thread queue mutex request with the corresponding index number first among the at least two candidate thread queue mutex requests; and use the thread identifier corresponding to the intermediate thread queue mutex request as the request information in the first thread queue mutex tracker to obtain the adjusted first thread queue mutex tracker.

[0206] In some embodiments of this application, the resource allocation device 600 further includes: an adjustment module, configured to change the occupancy status of the adjusted first thread queue mutex tracker from unoccupied to occupied in response to obtaining the adjusted first thread queue mutex tracker.

[0207] In some embodiments of this application, the first processing module 601 is further configured to, when there is a second thread queue mutex tracker with an occupied state in at least one thread queue mutex tracker, determine a second thread queue mutex request carrying a second thread queue mutex identifier in the thread queue mutex request set; wherein, the second thread queue mutex identifier is the mutex identifier corresponding to the second thread queue mutex tracker; determine a consistency determination result based on the thread identifier corresponding to the second thread queue mutex request and the current identifier corresponding to the request information in the second thread queue mutex tracker; the consistency determination result is used to characterize whether the second thread queue mutex request and the request in the second thread queue mutex tracker are in the same thread queue of the same thread group; and adjust the request information in the second thread queue mutex tracker based on the consistency determination result to obtain an adjusted second thread queue mutex tracker.

[0208] In some embodiments of this application, the first processing module 601 is further configured to, when the consistency determination result indicates that the second thread queue mutual exclusion request and the request in the second thread queue mutual exclusion tracker are in the same thread queue of the same thread group, use the thread identifier corresponding to the second thread queue mutual exclusion request and the current identifier together as the request information in the second thread queue mutual exclusion tracker to obtain the adjusted second thread queue mutual exclusion tracker; when the consistency determination result indicates that the second thread queue mutual exclusion request and the request in the second thread queue mutual exclusion tracker are not in the same thread queue of the same thread group, keep the current identifier corresponding to the request information in the second thread queue mutual exclusion tracker unchanged to obtain the second thread queue mutual exclusion tracker.

[0209] In some embodiments of this application, the second processing module 602 is further configured to determine, in the adjusted at least one thread queue mutex tracker, the target tracker whose corresponding occupancy status is occupied and whose corresponding allocation status is unallocated.

[0210] In some embodiments of this application, the third processing module 603 is further configured to, based on the allocation request, concatenate the N computing resources associated with the N thread blocks to obtain the target computing resource; allocate the target computing resource to N time-sequential target threads for synchronous use; wherein, the thread queue associated with the target thread identifier includes: N time-sequential target threads.

[0211] In some embodiments of this application, the resource allocation device 600 further includes: an adjustment module, used to change the allocation state of the target tracker from unallocated to allocated; wherein, the computing resources corresponding to the target tracker in the allocated state are unavailable to other threads.

[0212] In some embodiments of this application, the adjustment module is further configured to, in response to receiving mutual exclusion release requests from each target thread, stop allocating target computing resources to N target threads; adjust and reset the allocation state corresponding to the target tracker to unallocated; and reset the occupancy state of the target tracker to unoccupied.

[0213] It should be noted that the description of the above device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0214] It should be noted that, in the embodiments of this application, if the above-described resource allocation method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a terminal, server, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0215] Correspondingly, this application embodiment further provides a computer program product, which includes computer executable instructions. After the computer executable instructions are executed, they can implement the resource allocation method provided in this application embodiment.

[0216] Accordingly, this application embodiment further provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the resource allocation method provided in the above embodiments.

[0217] The descriptions of the above computer program product and storage medium embodiments are similar to those of the above method embodiments, and have similar technical descriptions and beneficial effects. Due to space limitations, please refer to the descriptions of the above method embodiments; therefore, they will not be repeated here. For technical details not disclosed in the computer program product and storage medium embodiments, please refer to the descriptions of the method embodiments of this application for understanding.

[0218] It should be understood that the phrase "an embodiment" or "one embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present application. Therefore, "in one embodiment" or "one embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of the present application, the sequence number of the above-described processes does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0219] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0220] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0221] Furthermore, in the embodiments of this application, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium, and when executed, it performs the steps of the above method embodiments.

[0222] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application.

Claims

1. A resource allocation method, characterized by, The resource allocation method is applied to a stream multiprocessor running N thread blocks, and the method comprises the following steps: In response to receiving a thread queue mutual exclusion request set, based on thread identifiers corresponding to the thread queue mutual exclusion request set, adjusting request information in at least one thread queue mutual exclusion tracker matched with the thread queue mutual exclusion request set by using a mutual exclusion lock mechanism, to obtain an adjusted at least one thread queue mutual exclusion tracker; the thread queue mutual exclusion request set comprises mutual exclusion requests of M threads belonging to different thread queues sent by each thread block, each thread queue is composed of N threads in time sequence, and the N threads correspond to N thread blocks respectively; In the adjusted at least one thread queue mutual exclusion tracker, a target tracker satisfying a resource allocation condition is determined; Based on a target thread identifier corresponding to an allocation request in the target tracker, an operation resource associated with the N thread blocks is allocated to a thread queue associated with the target thread identifier; The adjusting of the request information in the at least one thread queue mutual exclusion tracker matched with the thread queue mutual exclusion request set by using the mutual exclusion lock mechanism based on the thread identifiers corresponding to the thread queue mutual exclusion request set comprises the following steps: In a case where a second thread queue mutual exclusion tracker corresponding to an occupied state being occupied exists, a second thread queue mutual exclusion request carrying a second thread queue mutual exclusion identifier is determined in the thread queue mutual exclusion request set; the second thread queue mutual exclusion identifier is a mutual exclusion identifier corresponding to the second thread queue mutual exclusion tracker; Based on a consistency determination result between a thread identifier corresponding to the second thread queue mutual exclusion request and a current identifier corresponding to the request information in the second thread queue mutual exclusion tracker, the request information in the second thread queue mutual exclusion tracker is adjusted to obtain an adjusted second thread queue mutual exclusion tracker; the consistency determination result is used to represent whether the second thread queue mutual exclusion request and the request in the second thread queue mutual exclusion tracker are in the same thread queue.

2. The resource allocation method of claim 1, wherein, The adjusting of the request information in the at least one thread queue mutual exclusion tracker matched with the thread queue mutual exclusion request set by using the mutual exclusion lock mechanism based on the thread identifiers corresponding to the thread queue mutual exclusion request set comprises the following steps: In response to receiving the thread queue mutual exclusion request set, the at least one thread queue mutual exclusion tracker is determined; Based on the thread identifiers corresponding to the thread queue mutual exclusion request set, the request information in the at least one thread queue mutual exclusion tracker is adjusted by using the mutual exclusion lock mechanism to obtain the adjusted at least one thread queue mutual exclusion tracker; One of the thread queue mutual exclusion trackers corresponds to one thread queue, and the request information in the thread queue mutual exclusion tracker is used to record a request state of the corresponding thread queue.

3. The resource allocation method of claim 2, wherein, The determining of the at least one thread queue mutual exclusion tracker in response to receiving the thread queue mutual exclusion request set comprises the following steps: In response to receiving the thread queue mutual exclusion request set, index numbers corresponding to thread queue mutual exclusion requests in the thread queue mutual exclusion request set are determined; According to the order corresponding to the index number, the thread queue mutual exclusion tracker corresponding to each thread queue mutual exclusion request is determined based on the thread queue mutual exclusion identifier carried by the thread queue mutual exclusion request in sequence, and at least one thread queue mutual exclusion tracker corresponding to the thread queue mutual exclusion request set is obtained.

4. The resource allocation method of claim 3, wherein, The determining the index number corresponding to each thread queue mutual exclusion request in the thread queue mutual exclusion request set in response to receiving the thread queue mutual exclusion request set comprises: In response to receiving the thread queue mutual exclusion request set, the first identifier of each thread queue mutual exclusion request is determined; the first identifier is the unique identifier of the thread block corresponding to the thread queue mutual exclusion request; The second identifier of each thread queue mutual exclusion request is determined; the second identifier is the request sequence number of the thread queue mutual exclusion request in the request set in the thread block; the request set in the thread block includes the sequence number of the thread request in the thread block, and the request set in the thread block is all thread queue mutual exclusion requests sent by the thread block corresponding to the thread queue mutual exclusion request; Based on the first identifier and the second identifier, the index number corresponding to each thread queue mutual exclusion request is generated.

5. The resource allocation method of claim 4, wherein, The adjustment of the request information in the at least one thread queue mutual exclusion tracker matched with the thread queue mutual exclusion request set based on the thread identifier corresponding to the thread queue mutual exclusion request set by using the mutual exclusion lock mechanism to obtain the adjusted at least one thread queue mutual exclusion tracker further comprises: In the case that there is a first thread queue mutual exclusion tracker corresponding to the unoccupied state, a first thread queue mutual exclusion request carrying a first thread queue mutual exclusion identifier is determined in the thread queue mutual exclusion request set; wherein the first thread queue mutual exclusion identifier is the mutual exclusion identifier corresponding to the first thread queue mutual exclusion tracker; The thread identifier corresponding to the first thread queue mutual exclusion request is taken as the request information in the first thread queue mutual exclusion tracker to obtain the adjusted first thread queue mutual exclusion tracker.

6. The resource allocation method of claim 5, wherein, The first thread queue mutual exclusion request comprises at least two candidate thread queue mutual exclusion requests, and the thread identifier corresponding to the first thread queue mutual exclusion request is taken as the request information in the first thread queue mutual exclusion tracker to obtain the adjusted first thread queue mutual exclusion tracker, which comprises: Among the at least two candidate thread queue mutual exclusion requests, an intermediate thread queue mutual exclusion request whose index number is sorted in the front is selected; The thread identifier corresponding to the intermediate thread queue mutual exclusion request is taken as the request information in the first thread queue mutual exclusion tracker to obtain the adjusted first thread queue mutual exclusion tracker.

7. The resource allocation method of claim 5 or 6, wherein, The resource allocation method further comprises: In response to obtaining the adjusted first thread queue mutual exclusion tracker, the occupation state of the adjusted first thread queue mutual exclusion tracker is changed from unoccupied to occupied.

8. The method of claim 1, wherein, The adjustment of the request information in the second thread queue mutual exclusion tracker based on the consistency determination result between the thread identifier corresponding to the second thread queue mutual exclusion request and the current identifier corresponding to the request information in the second thread queue mutual exclusion tracker to obtain the adjusted second thread queue mutual exclusion tracker comprises: In a case where the consistency determination result indicates that the second thread team mutual exclusion request and the request in the second thread team mutual exclusion tracker are in the same thread team of the same thread team, the thread identification corresponding to the second thread team mutual exclusion request and the current identification are taken as the request information in the second thread team mutual exclusion tracker together, to obtain the adjusted second thread team mutual exclusion tracker; In a case where the consistency determination result indicates that the second thread team mutual exclusion request and the request in the second thread team mutual exclusion tracker are not in the same thread team of the same thread team, the current identification corresponding to the request information in the second thread team mutual exclusion tracker is kept unchanged, to obtain the second thread team mutual exclusion tracker.

9. The resource allocation method according to any one of claims 1 to 4, characterized by, The method further includes: In the adjusted at least one thread team mutual exclusion tracker, determining the target tracker that meets the resource allocation condition.

10. The resource allocation method of claim 9, wherein, In the adjusted at least one thread team mutual exclusion tracker, determining the target tracker that has an occupied state of being occupied and an allocation state of being unallocated. The method further includes: Based on the allocation request corresponding to the target thread identification in the target tracker, allocating the operation resources associated with the N thread blocks to the thread team associated with the target thread identification. Based on the allocation request, splicing the N operation resources associated with the N thread blocks to obtain target operation resources; 11. The resource allocation method of claim 10, wherein, Allocating the target operation resources to the N target threads for synchronous use in time sequence. The thread team associated with the target thread identification includes the N target threads in time sequence. The method further includes:

12. The resource allocation method of claim 10 or 11, wherein, Changing the allocation state of the target tracker from the unallocated state to the allocated state. The computing resources of the target tracker in the allocated state are unavailable to other thread teams. The method further includes:

13. A resource allocation apparatus, characterized by comprising: In response to receiving mutual exclusion release requests of the target threads, stopping the allocation of the target operation resources to the N target threads; Adjusting and resetting the allocation state of the target tracker to the unallocated state, and resetting the occupied state of the target tracker to the unoccupied state. The resource allocation apparatus is applied to a stream multiprocessor running N thread blocks, and includes: A first processing module is configured to, in response to receiving a thread team mutual exclusion request set, adjust, based on thread identifications corresponding to the thread team mutual exclusion request set, request information in at least one thread team mutual exclusion tracker matched by the thread team mutual exclusion request set by using a mutual exclusion lock mechanism, to obtain adjusted at least one thread team mutual exclusion tracker; the thread team mutual exclusion request set includes mutual exclusion requests of M threads belonging to different thread teams sent by each thread block, each thread team is composed of N threads in time sequence, and the N threads correspond to N thread blocks respectively; A second processing module is configured to determine, in the adjusted at least one thread team mutual exclusion tracker, a target tracker that meets a resource allocation condition; A third processing module is configured to, based on an allocation request corresponding to a target thread identification in the target tracker, allocate operation resources associated with the N thread blocks to a thread team associated with the target thread identification. The first processing module is further configured to, in a case where the second thread queue mutual exclusion tracker corresponding to the occupation state of the occupied exists, determine a second thread queue mutual exclusion request carrying a second thread queue mutual exclusion identifier in the thread queue mutual exclusion request set; the second thread queue mutual exclusion identifier is a mutual exclusion identifier corresponding to the second thread queue mutual exclusion tracker; based on a consistency determination result between a thread identifier corresponding to the second thread queue mutual exclusion request and a current identifier corresponding to request information in the second thread queue mutual exclusion tracker, the request information in the second thread queue mutual exclusion tracker is adjusted to obtain an adjusted second thread queue mutual exclusion tracker; the consistency determination result is used to represent whether the second thread queue mutual exclusion request and the request in the second thread queue mutual exclusion tracker are in the same thread queue.

14. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored computer executable instructions, and the computer executable instructions, after being executed, can implement the resource allocation method in any of claims 1 to 12.

Images