Data processing method, graphics processor, storage medium and program product

Abstract

The embodiment of the invention discloses a data processing method, a graphics processor, a storage medium and a program product, and the data processing method comprises the steps: responding to a processing request of any target task, and adjusting a synchronization identifier in real time; the synchronization identifier represents the number of a plurality of thread blocks which correspond to the target task and are in a ready-to-execute state, and the ready-to-execute state represents that the corresponding thread block reaches the core of a single graphics processor and is in the ready-to-execute state; in response to matching of the synchronization identifier and a quantity threshold value, loading data required by the plurality of thread blocks during processing at the same time; and executing a calculation task of the corresponding thread block based on the loaded data to process the target task.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to, but is not limited to, the technical field of computer technology, and particularly relates to a data processing method, a graphics processing unit, a storage medium and a program product. BACKGROUND

[0002] A task source end synchronization mechanism is gradually developed in a graphics processing unit (GPU) architecture to improve the utilization of a memory subsystem (MSS) through locality characteristics.

[0003] The existing task source end synchronization mechanism is a shared memory mechanism implemented within a single GPU thread block cluster. It allows different thread blocks in the same thread block cluster to share each other's shared memory, thereby improving data access speed and communication efficiency. However, this synchronization mechanism still has a lot of room for improvement, although it reduces data communication overhead. SUMMARY

[0004] Therefore, the embodiments of the present disclosure at least provide a data processing method, a graphics processing unit, a device, a storage medium and a program product.

[0005] The technical solutions of the embodiments of the present disclosure are implemented as follows: On the one hand, the embodiments of the present disclosure provide a data processing method, comprising: in response to a processing request of any target task, adjusting a synchronization identifier in real time; the synchronization identifier represents the number of a plurality of thread blocks corresponding to the target task and in a ready-to-execute state, and the ready-to-execute state represents that the corresponding thread block has reached the core of a single graphics processing unit and is in a ready-to-execute state; in response to the synchronization identifier matching a quantity threshold value, simultaneously loading data required by the plurality of thread blocks in processing; based on the loaded data, performing a computing task of the corresponding thread block to realize the processing of the target task.

[0006] In the embodiments of the present disclosure, in response to the fact that part or all of the threads corresponding to the target task have arrived at the GPU core, data of the arrived thread blocks is loaded at the same time, and the loaded data can be used multiple times in the GPU core, thereby maximizing the cache utilization rate; if the single GPU is an independent GPU, the loaded data can be used multiple times in multiple GPU cores in the GPU, thereby improving the cache utilization rate in the GPU; if all the GPUs in the GPU cluster are synchronized for data loading, the cache utilization rate in the GPU cluster can be doubled; and the start time of the synchronized data loading is also aligned with the calculation synchronization, so that the data is not constantly swapped in and out of the cache due to the asynchronization between the loading and the calculation, thereby improving the data utilization rate. Furthermore, by expanding the scope of the synchronization, the data communication overhead is minimized, and the overall utilization rate is improved.

[0007] In some embodiments, the data threshold is equal to the number of thread blocks corresponding to the target task.

[0008] In some embodiments, in the case where the number threshold is less than the number of thread blocks corresponding to the target task, the data processing method further comprises: after loading the data required by the multiple thread blocks during processing in response to the fact that the synchronization identifier matches the number threshold, resetting the synchronization identifier to the initial value, and repeating the steps of adjusting the synchronization identifier in real time, loading the data required by the multiple thread blocks during processing in response to the fact that the synchronization identifier matches the number threshold, and resetting the synchronization identifier to the initial value until the data loading of all the thread blocks corresponding to the target task is completed.

[0009] Based on the above technical solution, the data loading of the target task is divided into multiple synchronizations, one synchronization can realize the data loading of part of the thread blocks corresponding to the target task, and the performance improvement is only second to full synchronization, which is suitable for target tasks containing large-scale thread blocks.

[0010] In some embodiments, the above real-time adjustment of the synchronization identifier comprises: in response to each thread block that has arrived at the core of the single GPU and is in a ready-to-execute state, increasing the identifier value of the synchronization identifier by 1.

[0011] Based on the above technical solution, by adjusting the synchronization identifier in real time, the number of thread blocks that have arrived at the GPU core can be learned in time.

[0012] In some embodiments, the data processing method further comprises: recording the waiting time of the thread block in the ready-to-execute state that first arrives at the core of the single graphics processor; and loading data of a plurality of thread blocks corresponding to the synchronization identifier simultaneously in a case where the waiting time exceeds a time threshold.

[0013] Based on the above technical solution, the time threshold can prevent deadlock, although the determination of the time threshold does not achieve full synchronization of all thread blocks, but the maximum number of thread block synchronization has been achieved, and the performance improvement is only second to full synchronization.

[0014] In some embodiments, the above simultaneous loading of data required by the plurality of thread blocks during processing comprises: simultaneously loading data of a plurality of thread blocks from different thread block clusters from a memory system.

[0015] Based on the above technical solution, simultaneously loading data of a plurality of thread blocks from different thread block clusters can achieve synchronous loading of data of different thread block clusters.

[0016] In some embodiments, the above simultaneous loading of data of a plurality of thread blocks from different thread block clusters from a memory system comprises: when data of any first thread block in the plurality of thread blocks is initially loaded, obtaining the data of the first thread block from a dynamic random access memory of the memory system and reading out through a multi-level cache structure of the memory system; when data of any second thread block in the plurality of thread blocks is not initially loaded, reading out the data of the second thread block from a target level cache structure of the memory system; and the data of the second thread block is cached to the target level cache structure when it is initially loaded.

[0017] Based on the above technical solution, when the data of the thread block is initially loaded, the data is cached to a target level cache structure (such as a second level cache (L2) / last level cache (LLC)), so that when the same data is obtained by the thread block later, it does not need to be read from the DRAM, but can be directly read from the L2 / LLC, which can reduce data communication overhead and reduce the overall loading time of the memory subsystem, thereby obtaining the benefits of L2 / LLC in some scenarios of artificial intelligence computing.

[0018] In some embodiments, the execution code of the target task is added with code for executing the steps of the above method.

[0019] Based on the above technical solution, the execution code of the target task is added with code for executing the steps of the above method, in order to ensure that the target task is loaded and calculated synchronously according to the above data processing method, to minimize the loading time of data, improve data utilization, and thereby reduce data communication overhead.

[0020] In another aspect, the embodiments of the present disclosure provide a graphics processor, the graphics processor stores a computer program capable of being executed, and the graphics processor implements part or all of the steps in the above method when executing the program.

[0021] In another aspect, the embodiments of the present disclosure provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement part or all of the steps in the above method.

[0022] In another aspect, the embodiments of the present disclosure provide a computer program, which includes computer readable code, and when the computer readable code is executed in a computer device, a processor in the computer device executes part or all of the steps for implementing the above method.

[0023] In another aspect, the embodiments of the present disclosure provide a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, and when the computer program is read and executed by a computer, part or all of the steps in the above method are implemented.

[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the technical solutions of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0025] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the technical solutions of the present disclosure.

[0026] Figure 1 An implementation flowchart of a data processing method provided by the embodiments of the present disclosure Figure 1 ; Figure 2 A schematic diagram of an existing synchronization scheme provided by the embodiments of the present disclosure Figure 3 A synchronization scheme of a data processing method provided by the embodiments of the present disclosure Figure 4 An implementation flowchart of a data processing method provided by the embodiments of the present disclosure Figure 2 ; Figure 5 An implementation flowchart of a data processing method provided by the embodiments of the present disclosure Figure 3 ; Figure 6 An implementation flowchart of a data processing method provided by the embodiments of the present disclosure Figure 4 ; Figure 7 A component structure schematic diagram of a data processing device provided by the embodiments of the present disclosure. DETAILED DESCRIPTION

[0027] In order to make the purposes, technical solutions and advantages of the present disclosure clearer, the technical solutions of the present disclosure are described in further detail below in combination with the drawings and embodiments, and the described embodiments should not be regarded as limitations of the present disclosure. All other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present disclosure.

[0028] In the following description, "some embodiments" are related to a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0029] The terms "first / second / third" involved are only to distinguish similar objects and do not represent a specific order of the objects. It can be understood that "first / second / third" can interchange specific order or sequence as allowed, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. The terms used herein are only for the purpose of describing the present disclosure and are not intended to limit the present disclosure.

[0031] The embodiments of the present disclosure provide a data processing method, which can be applied to a graphics processor, for example, a single graphics processor. The single graphics processor refers to any one graphics processor in a graphics processor cluster. As shown in the figure, the method comprises the following steps 101 to 103: Figure 1 Step 101, in response to a processing request of any target task, adjusting a synchronization identifier in real time; the synchronization identifier represents the number of multiple thread blocks corresponding to the target task in a ready-to-execute state, and the ready-to-execute state represents that the corresponding thread block has reached the core of the single graphics processor and is in a ready-to-execute state.

[0032] The graphics processor cluster refers to a GPU cluster. The processing request refers to a request for processing a target task. The target task refers to any task that the graphics processor can implement. For example, the target task can be a tensor data calculation task, an image processing task, a floating point calculation task, and a model training task, etc. The processing of a task requires multiple thread blocks. A thread block is a set of threads, and the multiple thread blocks can be threads in the same thread block cluster or threads in multiple thread block clusters.

[0033] ​The synchronization identifier is used to represent the number of thread blocks that have arrived at a core of a graphics processing unit (GPU). For example, the synchronization identifier can be represented as a flag. Different thread blocks can be executed on different GPU cores. The ready-to-execute state is used to represent that the corresponding thread block has arrived at the GPU core (e.g., the thread block has been scheduled to the GPU core), is in the ready-to-execute state, but has not yet been executed.

[0034] In some embodiments, the implementation of step 101 can be as follows: before the target task starts processing, the synchronization identifier is set to an initial value; and then, each time a thread block arrives at a core of a graphics processing unit, the specific value of the synchronization identifier is incremented by one.

[0035] Specifically, an atomic operation function can be used to increment the specific value of the synchronization identifier by one each time a thread block arrives at a core of a graphics processing unit. For example, the atomic operation function can be atomicAdd(flag, 1).

[0036] For example, when the target task starts processing (workgroup start), the synchronization identifier flag is initialized to 0; and then, each time a thread block arrives at a core of a graphics processing unit (GPU), the flag is adjusted as follows: flag = flag + 1. That is, for the first thread block that arrives at the GPU core, the flag is adjusted as follows: flag = 0 + 1 = 1; for the second thread block that arrives at the GPU core, the flag is adjusted as follows: flag = 1 + 1 = 2; and so on.

[0037] Step 102: in response to the synchronization identifier matching the number threshold, data required by the plurality of thread blocks for processing is loaded at the same time.

[0038] The number threshold is used to determine the timing of data loading. The synchronization identifier matching the number threshold, for example, includes the value of the synchronization identifier being equal to the number threshold.

[0039] In some embodiments, the data threshold is equal to the number of thread blocks corresponding to the target task. At this time, the number threshold is used to determine whether all thread blocks corresponding to the target task have arrived at a core of a graphics processing unit (GPU). The synchronization identifier matching the number threshold means that all thread blocks corresponding to the target task have arrived at the GPU core.

[0040] In some embodiments, the number threshold is less than the number of thread blocks corresponding to the target task. At this time, the number threshold is used to determine whether all thread blocks corresponding to the target task have arrived at a core of a graphics processing unit (GPU). The synchronization identifier matching the number threshold means that some thread blocks corresponding to the target task have arrived at the GPU core.

[0041] In some embodiments, the quantity threshold can be carried in the processing request when the target task is issued. Or, the quantity threshold corresponding to the target task can be pre-stored in the graphics processor.

[0042] It should be noted that in the case that all / partial threads corresponding to the target task have arrived at the GPU core, the data of the thread block is loaded at the same time; thus, the start time of the synchronized data loading is also aligned with the calculation synchronization, the loaded data can be used multiple times in the GPU core, maximizing the cache utilization rate; and the data utilization rate will not be extremely low due to the continuous swapping of data in and out of the cache caused by the asynchronization of loading and calculation.

[0043] Step 103, performing the calculation task of the corresponding thread block based on the loaded data, to realize the processing of the target task.

[0044] In some embodiments, the data threshold is equal to the number of thread blocks corresponding to the target task. In this case, in response to the processing request of the target task, the value of the synchronization identifier is incremented by one each time a thread block arriving at a core of a single graphics processor is determined, and in the case that the synchronization identifier matches the quantity threshold, the data required by all thread blocks corresponding to the target task during processing is loaded at the same time, the calculation task of the corresponding thread block is performed based on the data of all thread blocks corresponding to the target task, and the processing of the target task is realized.

[0045] In some embodiments, the quantity threshold is less than the number of thread blocks corresponding to the target task. In this case, after the data required by the plurality of thread blocks during processing is loaded at the same time in response to the synchronization identifier matching the quantity threshold, the synchronization identifier is reset to the initial value, and the steps of adjusting the synchronization identifier in real time, loading the data required by the plurality of thread blocks during processing at the same time in response to the synchronization identifier matching the quantity threshold, and resetting the synchronization identifier to the initial value are repeated until the data of all thread blocks corresponding to the target task is loaded. In this case, the number of the last batch of thread blocks can be less than the quantity threshold. For example, the quantity threshold is 5, and the target task has a total of 12 thread blocks, then the last batch has only 2 thread blocks, and in the case that it is judged that this is the last 2 thread blocks of the target task, it can be considered that the synchronization identifier matches the quantity threshold, and the data required by the plurality of thread blocks during processing is loaded at the same time.

[0046] In some embodiments, in response to a processing request of a target task, a value of a synchronization identifier is incremented by one each time a thread block arrives at a core of a single graphics processor, and in a case where the synchronization identifier matches a quantity threshold, data required by a plurality of thread blocks corresponding to the synchronization identifier for processing is loaded simultaneously, the plurality of thread blocks being partial thread blocks corresponding to the target task; the synchronization identifier is set to an initial value, and for other threads that are not loaded, the synchronization identifier is adjusted in real time based on a case where the other threads arrive at the core of the single graphics processor, and data of the other thread blocks is loaded simultaneously in a case where the other threads have all arrived at the core of the single graphics processor; and a computing task of each of the thread blocks is executed based on data of the thread block, so as to implement processing of the target task.

[0047] In some embodiments, the synchronization identifier can be set to the initial value at any time before the target task starts, so as to ensure that the first thread block arrives at the core of the single graphics processor before the synchronization identifier is set to the initial value. For example, the initial value can be 0.

[0048] The target task starts processing before at least one of the following: after the synchronization identifier of a previous task matches the quantity threshold, after the previous task processing is completed, and when the target task starts processing.

[0049] As shown in FIG. 1, a shared memory mechanism implemented in a single GPU thread block cluster (Thread Block Cluster) in the prior art, each thread block (Thread Block) in the same Thread Block Cluster has its own shared memory (shared memory), and these shared memories can read and write and communicate between different Thread Blocks in the same Cluster, but cannot realize read and write and communication between different Clusters. Figure 2

[0050] As shown in FIG. 2, embodiments of the present disclosure can load data from Thread Blocks in different Thread Block Clusters simultaneously by synchronizing the start time of data loading of a plurality of thread blocks corresponding to a target task, so as to expand the scope of sync to read and write and communication between Clusters, for example, to obtain the benefits of L2 / LLC, etc. (i.e., efficiency improvement of data communication) in some scenarios of AI computing, so as to minimize data communication overhead, reduce the overall load time of the memory subsystem, and improve the overall utilization. Figure 3

[0051] ​​In the embodiments of the present disclosure, in response to a processing request of any target task, the synchronization identifier is adjusted in real time based on the situation that the thread blocks corresponding to the target task arrive at the cores of a single graphics processor; in response to the synchronization identifier matching a quantity threshold, data required by the multiple thread blocks in processing is simultaneously loaded; and the calculation task of the corresponding thread blocks is executed based on the loaded data to realize the processing of the target task. In this way, in the case that all or part of the threads corresponding to the target task have arrived at the cores (GPU cores) of a single graphics processor, the data of the thread blocks is simultaneously loaded, the loaded data can be used multiple times in the GPU cores within the single graphics processor, and the cache utilization rate is maximized; if the single graphics processor is a GPU that exists independently, the loaded data can be used multiple times in the multiple cores GPU cores within the GPU, and the cache utilization rate within the GPU is improved; if all the GPUs within the GPU cluster are synchronized in data loading, the cache utilization rate within the GPU cluster can be doubled. Moreover, the start time of the synchronized data loading is also aligned with the synchronization of the calculation, and the data utilization rate is improved because the data is not constantly swapped in and out of the cache due to the asynchronization of the load and the calculation. Further, by expanding the scope of the synchronization (sync), for example, in some scenarios of artificial intelligence (AI) calculation, the benefits of the second-level cache / last-level cache, etc. can be obtained, so as to maximize the reduction of the data communication overhead, the overall load time of the memory subsystem (MSS), and the overall utilization rate.

[0052] The embodiments of the present disclosure provide a data processing method, which can be applied to a graphics processor. As shown in the method includes the following steps 401 to 405: Figure 4 Step 401, in response to a processing request of any target task, a synchronization identifier is adjusted in real time; the synchronization identifier represents the quantity of multiple thread blocks corresponding to the target task in a ready-to-execute state, and the ready-to-execute state represents that the corresponding thread blocks have arrived at the cores of a single graphics processor and are in a ready-to-execute state.

[0053] In some embodiments, whether the data threshold is equal to the quantity of the thread blocks corresponding to the target task or less than the quantity of the thread blocks corresponding to the target task, the following can be performed: in response to each thread block arriving at the cores of the single graphics processor and being in the ready-to-execute state, the identifier value of the synchronization identifier is increased by 1.

[0054] Step 402, in response to the synchronization identifier matching a quantity threshold, data required by the multiple thread blocks in processing is simultaneously loaded.

[0055] ​Here, steps 401-402 correspond to the aforementioned step 101, and the specific implementation of steps 401-402 can refer to the specific implementation of the aforementioned step 101.

[0056] In some embodiments, the specific implementation of step 402 can be: simultaneously loading data of the plurality of thread blocks from different cluster of thread blocks from the memory system.

[0057] The memory system can be a MMS system. The memory system can include a multi-level cache mechanism and a dynamic random access memory (DRAM). The dynamic random access memory can be a double data rate synchronous dynamic random memory (DDR).

[0058] In some embodiments, the Inline Atomic Synchronization Instructions in the kernel corresponding to the target task can be invoked to achieve the cross-cluster loading synchronization and the calculation synchronization of the entire GPU.

[0059] Step 403: based on the loaded data, performing the calculation task of the corresponding thread block to achieve the processing of the target task.

[0060] In some embodiments, the data processing method provided by the embodiments of the present disclosure further includes steps 404-405: Step 404: recording the waiting time of the thread block that first arrives at the core of the single graphics processor and is in a state ready for execution.

[0061] Step 405: in the case where the waiting time exceeds a time threshold, simultaneously loading data of a plurality of thread blocks corresponding to the synchronization identifier.

[0062] The plurality of thread blocks corresponding to the synchronization identifier refers to the plurality of thread blocks that have currently arrived at the core of the single graphics processor. The waiting time refers to the length of time from the time when the thread block arrives at the core of the single graphics processor to the current time. The time threshold is used to prevent deadlock. The time threshold can be determined according to the physical performance of the GPU and the processing time of the task. The time threshold can be represented as timeout. In a feasible implementation, the value of the time threshold can be determined through multiple experiments to achieve the optimal effect through tuning.

[0063] In some embodiments, when the waiting time exceeds the time threshold, the data of the multiple thread blocks corresponding to the synchronization identifier is loaded simultaneously (even though the value of the synchronization identifier may not have reached the quantity threshold at this time), and the computing task of the corresponding thread block is executed based on the loaded data; the thread block that has not arrived at the GPU core can perform data loading and computing in the next round. That is, either the waiting time exceeding the time threshold or the synchronization identifier matching the quantity threshold can trigger the simultaneous loading of the data required by the multiple thread blocks in processing.

[0064] According to the embodiments of the present disclosure, the number of thread blocks that have arrived at the GPU core can be learned in real time by adjusting the synchronization identifier in real time, and the synchronization time of data loading can be determined through the matching relationship between the synchronization identifier and the quantity threshold. The time threshold can prevent deadlock, although the determination of the time threshold does not achieve full synchronization of all thread blocks, but the maximum number of thread blocks is synchronized, and the performance improvement is only second to full synchronization.

[0065] The embodiments of the present disclosure provide a data processing method, which can be applied to a graphics processor. As shown in the figure, the method comprises the following steps 501 to 503: Figure 5 Step 501, in response to a processing request of any target task, adjusting a synchronization identifier in real time; the synchronization identifier represents the number of multiple thread blocks corresponding to the target task in a ready-to-execute state, and the ready-to-execute state represents that the corresponding thread block has arrived at the core of a single graphics processor and is in a ready-to-execute state.

[0066] Step 502, in response to the synchronization identifier matching a quantity threshold, simultaneously loading the data of the multiple thread blocks from different thread block clusters from a memory system.

[0067] In some embodiments, the specific implementation of step 502 can be: when the data of any first thread block in the multiple thread blocks is initially loaded, obtaining the data of the first thread block from a dynamic random access memory of the memory system and reading out through a multi-level cache structure of the memory system.

[0068] When the data is initially loaded, the data is read from the dynamic random access memory (DRAM) and cached in the multi-level cache structure, so that other thread blocks obtain data from the multi-level cache structure, improve the utilization of data, and reduce the data communication overhead.

[0069] ​In some embodiments, the implementation of step 502 can also include: reading out the data of any second thread block in the plurality of thread blocks from a target level cache structure of the memory system when the data of the second thread block is not initially loaded; and caching the data of the second thread block to the target level cache structure when the data of the second thread block is initially loaded.

[0070] The target level cache structure refers to a cache structure in which the data of the second thread block is stored. For example, the target level cache structure can be L2 / LLC.

[0071] Since the data of the thread block is cached to the second level cache (L2) / last level cache (LLC) when initially loaded, when the same data is obtained by the thread block later, the data does not need to be read from the DRAM again, but can be directly read from the L2 / LLC, which can reduce the data communication overhead, reduce the overall loading time of the memory subsystem (MSS), and thus obtain the benefits of L2 / LLC in some scenarios of artificial intelligence (AI) calculation.

[0072] Step 503: performing a calculation task of the corresponding thread block based on the loaded data to implement processing of the target task.

[0073] Here, the above steps 501 to 503 correspond to the above steps 101 to 103 or steps 401 to 403 respectively, and in implementation, the specific embodiments of the above steps 101 to 103 or steps 401 to 403 can be referred to.

[0074] In some embodiments, the data processing method provided by the embodiments of the present disclosure can further include: adding code for performing the steps of the above data processing method in the execution code of the target task.

[0075] In some embodiments, when the processing request of the target task is received, the operation of adjusting the synchronization identifier in real time, the operation of determining whether the synchronization identifier matches the threshold value of the number of thread blocks, and the operation of determining whether the waiting time exceeds the time threshold value are added to the execution code of the target task to perform the steps of the above data processing method.

[0076] In some embodiments, the operation of adjusting the synchronization identifier in real time, the operation of determining whether the synchronization identifier matches the threshold value of the number of thread blocks, and the operation of determining whether the waiting time exceeds the time threshold value can also be set by the user in the processing request, so that the GPU can perform the operation of adjusting the synchronization identifier in real time, the operation of determining whether the synchronization identifier matches the threshold value of the number of thread blocks, and the operation of determining whether the waiting time exceeds the time threshold value before performing the data loading and calculation operations after receiving the processing request.

[0077] In the embodiments of the present disclosure, the operation of adjusting the synchronization identifier in real time, the operation of determining whether the synchronization identifier matches the thread block quantity threshold, and the operation of determining whether the waiting time exceeds the time threshold can be set by a user when processing a request, can be added to the execution code of the target task when the processing request is received, or can also be pre-set in the execution code of the target task.

[0078] The application of the data processing method provided by the embodiments of the present disclosure in an actual scenario is described below.

[0079] The core principle of the embodiments of the present disclosure is to control the start time of the loaded data of the computing engine (for example: tensor) to achieve synchronization. Because each calculation needs to load data and then perform calculation, the start time of the synchronized loading will also align the synchronization of the calculation. The loaded data will be used multiple times in multiple GPU cores, maximizing the cache utilization rate. Instead of causing data to be constantly swapped in and out of the cache due to asynchronous loading and calculation, resulting in extremely low data utilization.

[0080] As shown in FIG. 6, the specific process of synchronization through the atomic mechanism includes: Figure 6 Step 601, initialize flag as 0 by workgroup start. Here, workgroup start refers to the start of the target task processing. flag represents the number of thread blocks that have reached the GPU core.

[0081] Step 602, atomicAdd(flag, 1).

[0082] Step 603, while (flag!= block_num) || not timeout.

[0083] Here, flag!= block_num indicates that the number of thread blocks that have reached the GPU core does not match block_num (the quantity threshold). while (flag!= block_num) || not timeout indicates that when the number of thread blocks that have reached the GPU core does not match block_num, and the waiting time of the thread block does not exceed timeout (the time threshold), step 603 is continued to be executed; when the number of thread blocks that have reached the GPU core matches block_num, and / or the waiting time of the thread block exceeds timeout, step 604 is executed.

[0084]

[0085] ​timeout is a mechanism to prevent deadlock. The value of timeout needs to be tuned to achieve the best result; before timeout, the flag has achieved the maximum number of block synchronization, and the performance improvement is between zero synchronization and full synchronization.

[0086] Step 604, do load & compute.

[0087] Step 605, whether the kernel is executed.

[0088] If the kernel is not executed, go to step 602; if the kernel is executed, go to step 606.

[0089] Step 606, workgroup end.

[0090] The synchronization scheme of the embodiment of the present disclosure is simple to implement, and only needs to add the synchronization code of steps 601 to 603 in the load and compute code of each kernel workgroup that needs to be synchronized to improve performance, so as to achieve the maximum utilization of MSS brought by the synchronization mechanism.

[0091] The embodiment of the present disclosure achieves the load / compute synchronization scheme across clusters of the whole GPU by the way of inline atomic synchronization instructions (Inline Atomic Synchronization Instructions) in the calling task kernel. Since the synchronization across clusters (MPC) will bring more data reuse benefits. That is, the data used by one MPC will stay in L2 / LLC for a short time, and since the data will not be kicked out immediately in a short time, if another MPC also wants to use these data at the same time, the data can be reused. However, if there is no synchronization mechanism, the data will be kicked out due to replacement for a long time, so that when other MPCs want to use these data, the data has to be moved to L2 / LLC again. Therefore, the synchronization mechanism is implemented, which ensures the reuse in a short time and realizes the maximum locality at the L2 / LLC level, while other technical solutions only achieve the synchronization within the cluster.

[0092] In addition, the embodiment of the present disclosure fully considers the complex runtime environment, prevents deadlock through the timeout mechanism, and achieves the synchronization of part of the workgroup within the timeout.

[0093] Compared with the cluster.sync and distribute share memory scheme in the prior art, the present technology further expands the scope of sync to the entire GPU range across the cluster, and the locality benefit is not limited to share memory, but also to the global scope of L2 / LLC.

[0094] Based on the foregoing embodiments, the embodiments of the present disclosure provide a data processing apparatus, which comprises various units and modules included in the units, and can be implemented by a processor in a computer device. Of course, it can also be implemented by a specific logic circuit. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA).

[0095] Figure 7 The constituent structure of a data processing apparatus provided by the embodiments of the present disclosure is shown in FIG. 7, which comprises a processing module 710 and a loading module 720. Figure 7 The processing module 710 is configured to adjust a synchronization identifier in real time in response to a processing request of any target task, and the synchronization identifier represents the number of multiple thread blocks corresponding to the target task and in a ready-to-execute state. The ready-to-execute state represents that the corresponding thread block has reached the core of a single graphics processor and is in a ready-to-execute state. The loading module 720 is configured to load the data required by the multiple thread blocks in processing at the same time in response to the synchronization identifier matching a quantity threshold value. The processing module 710 is further configured to execute the computing task of the corresponding thread block based on the loaded data, and implement the processing of the target task.

[0096] In some embodiments, the data threshold value is equal to the number of thread blocks corresponding to the target task.

[0097] ​In some embodiments, when the number threshold is less than the number of thread blocks corresponding to the target task, the loading module 720 is further configured to: in response to the synchronization identifier matching the number threshold, simultaneously load data required by the plurality of thread blocks in processing, reset the synchronization identifier to an initial value, and repeatedly perform the steps of adjusting the synchronization identifier in real time, in response to the synchronization identifier matching the number threshold, simultaneously loading data required by the plurality of thread blocks in processing, and resetting the synchronization identifier to an initial value until data loading of all thread blocks corresponding to the target task is completed.

[0098] In some embodiments, the processing module 710 is further configured to: in response to each thread block arriving at the core of the single graphics processor in a ready-to-execute state, add 1 to the identification value of the synchronization identifier.

[0099] In some embodiments, the loading module 720 is further configured to: record the waiting time of the first thread block arriving at the core of the single graphics processor in a ready-to-execute state; and in response to the waiting time exceeding a time threshold, simultaneously load data of a plurality of thread blocks corresponding to the synchronization identifier.

[0100] In some embodiments, the loading module 720 is specifically configured to: simultaneously load data of the plurality of thread blocks from different thread block clusters from the memory system.

[0101] In some embodiments, the loading module 720 is specifically configured to: in response to data of any first thread block in the plurality of thread blocks being initially loaded, obtain data of the first thread block from a dynamic random access memory of the memory system and read out through a multi-level cache structure of the memory system.

[0102] In some embodiments, the loading module 720 is specifically configured to: in response to data of any second thread block in the plurality of thread blocks not being initially loaded, read out data of the second thread block from a target level cache structure of the memory system; and data of the second thread block is cached to the target level cache structure when initially loaded.

[0103] In some embodiments, the execution code of the target task is added with code for executing the steps of the above data processing method.

[0104] The above device embodiments are similar to the above method embodiments in description, and have similar beneficial effects as the method embodiments. In some embodiments, the device provided by the embodiments of the present disclosure has functions or includes modules that can be used to execute the methods described in the above method embodiments. For technical details not disclosed in the device embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure.

[0105] It should be noted that, in the embodiments of the present disclosure, if the data processing method described above is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present disclosure can be embodied in the form of a software product in essence or the part that contributes to the related art, which is stored in a storage medium, includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the methods described in the embodiments of the present disclosure. The storage medium mentioned above includes: a U disk, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, and various media that can store program codes. Thus, the embodiments of the present disclosure are not limited to any specific hardware, software or firmware, or any combination of hardware, software and firmware.

[0106] The embodiments of the present disclosure provide a graphics processor, and the graphics memory stores a computer program that can be run, and the processor implements part or all of the steps of the above method when executing the program.

[0107] The embodiments of the present disclosure provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement part or all of the steps of the above method. The computer readable storage medium can be transitory or non-transitory.

[0108] The embodiments of the present disclosure provide a computer program, which includes computer readable code, and when the computer readable code is run in a computer device, a processor in the computer device executes part or all of the steps of the above method.

[0109] The embodiments of the present disclosure provide a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, and when the computer program is read and executed by a computer, part or all of the steps of the above method are implemented. The computer program product can be implemented by hardware, software or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (Software Development Kit, SDK) and the like.

[0110] It should be noted that the above-mentioned description of the various embodiments is inclined to emphasize the differences between the various embodiments, and the same or similar parts can be referred to each other. The above description of the device, storage medium, computer program and computer program product embodiments is similar to the description of the method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the device, storage medium, computer program and computer program product embodiments of the present disclosure, please refer to the description of the method embodiments of the present disclosure.

[0111] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present disclosure. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present disclosure, the size of the sequence number of each step / process does not mean the order of execution, and the execution order of each step / process should be determined by its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure. The above sequence number of the embodiments of the present disclosure is only for description, not representing the advantages or disadvantages of the embodiments.

[0112] It should be noted that in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article or device including the element.

[0113] In several embodiments provided by the present disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0114] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units; they can be located in one place or distributed on multiple network units; and part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0115] In addition, each functional unit in each embodiment of the present disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware or in the form of hardware plus software functional units.

[0116] Those skilled in the art can understand that all or part of the steps of the above method embodiments can be completed by program instruction related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps including the above method embodiments when executed; and the foregoing storage medium includes mobile storage devices, read only memory (Read Only Memory, ROM), magnetic discs or optical discs, and various storage medium that can store program codes.

[0117] Alternatively, the integrated units of the present disclosure, if implemented in the form of software functional modules and sold or used as independent products, can also be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present disclosure. The foregoing storage medium includes mobile storage devices, ROM, magnetic discs or optical discs, and various storage medium that can store program codes.

[0118] The above is only an embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present disclosure, which should be covered within the protection scope of the present disclosure.

Claims

1. A data processing method, characterized in that: The data processing method includes: In response to a processing request of any target task, adjusting a synchronization flag in real time; the synchronization flag indicates the number of multiple thread blocks corresponding to the target task that are in a ready-to-execute state, wherein the ready-to-execute state indicates that the corresponding thread block has arrived at a core of a single graphics processor and is in a ready-to-execute state; In response to the synchronization identifier matching a quantity threshold, simultaneously loading data required by the plurality of thread blocks during processing; The computing task of the corresponding thread block is executed based on the loaded data to realize the processing of the target task.

2. The data processing method according to claim 1, wherein: The data threshold is equal to the number of thread blocks corresponding to the target task.

3. The data processing method according to claim 1, wherein: When the quantity threshold is less than the number of thread blocks corresponding to the target task, the data processing method further includes: After simultaneously loading the data required for processing the multiple thread blocks in response to the synchronization identifier matching the quantity threshold, the synchronization identifier is reset to an initial value, and the steps of adjusting the synchronization identifier in real time, simultaneously loading the data required for processing the multiple thread blocks in response to the synchronization identifier matching the quantity threshold, and resetting the synchronization identifier to the initial value are repeated until the data of all thread blocks corresponding to the target task are loaded.

4. The data processing method according to claim 1, wherein: The real-time adjustment of the synchronization flag includes: In response to each thread block arriving at the core of the single graphics processor and being in a ready-to-execute state, the identification value of the synchronization identification is increased by 1.

5. The data processing method according to claim 1, wherein: The data processing method further includes: Recording the waiting time of the first thread block that arrives at the core of the single graphics processor and is in a ready-to-execute state; When the waiting time exceeds a time threshold, data of multiple thread blocks corresponding to the synchronization identifiers are loaded simultaneously.

6. The data processing method according to claim 1, wherein: The simultaneously loading of data required by the plurality of thread blocks during processing includes: Data of the plurality of thread blocks from different thread block clusters are loaded simultaneously from the memory system.

7. The data processing method according to claim 6, characterized in that: The step of simultaneously loading data of the plurality of thread blocks from different thread block clusters from the memory system includes: When data of any first thread block among the plurality of thread blocks is loaded for the first time, obtaining the data of the first thread block from a dynamic random access memory of the memory system and reading the data through a multi-level cache structure of the memory system; When data of any second thread block among the plurality of thread blocks is not loaded for the first time, the data of the second thread block is read out from the target level cache structure of the memory system; the data of the second thread block is cached in the target level cache structure when loaded for the first time.

8. The data processing method according to claim 1, wherein: The execution code of the target task is added with codes for executing the steps of the method according to any one of claims 1 to 7.

9. A graphics processor storing an executable computer program, characterized in that: When the graphics processor executes the program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer program product, characterized in that The computer program product comprises a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, the steps of the method according to any one of claims 1 to 8 are implemented.

Images