Data processing method, parallel processor, electronic equipment and storage medium

Abstract

A data processing method, a parallel processor, an electronic device and a storage medium are applied to the field of data processing. The data processing method includes performing, by a parallel processor, computing processing operations including a plurality of computing operations sequentially performed, the computing processing operations being performed by the parallel processor, comprising the following steps: for each calculation operation, determining at least one competition work block in a plurality of work blocks according to the number of the plurality of work blocks and the number of a plurality of calculation units included in input data of a target kernel function corresponding to the calculation operation; processing the computing tasks of the plurality of working blocks in parallel in a pipelined manner by the plurality of computing units to complete computing operation; wherein for the at least one competitive work block, the distribution relation between the calculation unit and the calculation task of the at least one competitive work block is dynamically determined through a competitive mechanism. The method can eliminate the idle time accumulated by the long-tail effect, and improves the overall performance.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the field of data processing, and in particular to a data processing method, a parallel processor, an electronic device, and a non-transitory computer-readable storage medium. BACKGROUND

[0002] In the field of high-performance programming of parallel processors such as graphic processing units (GPUs), data parallelism is a core programming paradigm. Developers usually divide the data of a large-scale computing task (such as matrix multiplication, convolution, etc.) into multiple independent data blocks of the same shape, called work tiles. Subsequently, the parallel processor starts a large number of computing units to process these work tiles in parallel.

[0003] In distributed computing scenarios such as pre-training and post-training of large models, there is often a case of overlapping execution of communication and computation, that is, communication processing operations and computing processing operations are performed in parallel within the same time period. The relationship between communication and computation is embodied as data dependency and cooperation. The communication processing operation provides the distributed data required by the computation, and the computing processing operation performs local operations based on the data delivered by the communication. Alternatively, the communication processing operation transmits the data generated by the computing processing operation to a specified location. The communication processing operation and the computing processing operation form a pipeline-like cooperation through data flow.

[0004] Current data parallel processing may cause some computing units to be in an idle state in a certain wave when the number of work tiles and the number of computing units are not in an integer multiple relationship, reducing the utilization rate of the processor. And especially in the implementation mode of overlapping execution of communication and computation, this idling will cause the total idle time cost to increase exponentially, greatly reducing the performance of the computation and communication fusion. SUMMARY

[0005] The application provides a data processing method applied to a parallel processor, the parallel processor comprising a plurality of computing units, the data processing method comprising performing a computing processing operation by the parallel processor, wherein the computing processing operation comprises a plurality of computing operations executed sequentially, input data of the computing processing operation is divided into a plurality of data blocks, and each computing operation is used for performing processing of one data block in the plurality of data blocks, wherein performing the plurality of computing operations of the computing processing operation by the parallel processor comprises: for each computing operation, determining at least one competitive work block in a plurality of work blocks according to a number of the plurality of work blocks included in input data of a target kernel function corresponding to the computing operation and a number of the plurality of computing units; and processing, in a pipeline manner, computing tasks of the plurality of work blocks by the plurality of computing units to complete the computing operation; wherein for the at least one competitive work block, an allocation relationship between the plurality of computing units and the computing tasks of the at least one competitive work block is dynamically determined by a competition mechanism, the competition mechanism comprises that the plurality of computing units compete for execution qualifications of the computing tasks of the at least one competitive work block, and a computing unit obtaining the execution qualification executes a corresponding competitive work block.

[0006] For example, in the data processing method provided by at least one embodiment of the application, the at least one wave corresponding to the target kernel function comprises a competitive wave, and a work block in the competitive wave is used as the at least one competitive work block, for any computing unit in the plurality of computing units, after the computing unit completes a computing task of a target work block, the computing unit cyclically uses an atomic operation to compete for an execution qualification of the at least one competitive work block, wherein the target work block comprises a work block in a previous wave of a first competitive wave in the at least one wave; in response to that the any computing unit obtains the execution qualification of any competitive work block, the any computing unit executes the computing task of the any competitive work block; and in response to that the any computing unit does not obtain the execution qualification of any competitive work block, the any computing unit is scheduled to execute a subsequent processor task.

[0007] For example, in the data processing method provided in at least one embodiment of the present application, the allocation relationship between the plurality of computing units and the computing tasks of the at least one competition work block is dynamically determined through a competition mechanism, including: initializing a competition workspace, wherein the competition workspace includes R lock variables initialized in an unlocked state, R is a positive integer and represents the number of the at least one competition work block, and the R lock variables correspond one-to-one to the at least one competition work block; using an atomic operation to compete for the R lock variables in the competition workspace by the plurality of computing units in a loop to determine whether any lock variable can be updated from the unlocked state to a locked state; determining that a first computing unit that updates a lock variable to the locked state obtains the execution qualification of the computing task of the competition work block corresponding to the lock variable, and the first computing unit exits the competition and executes the computing task of the competition work block corresponding to the lock variable; determining that a second computing unit that does not update any lock variable to the locked state ends the execution operation of the target kernel function and schedules the second computing unit to execute a subsequent processor task.

[0008] For example, in the data processing method provided in at least one embodiment of the present application, using an atomic operation to compete for the R lock variables in the competition workspace by the plurality of computing units in a loop to determine whether any lock variable can be updated from the unlocked state to a locked state, including: for any computing unit in the plurality of computing units: loop through the R lock variables and perform the atomic operation with the R lock variables respectively, wherein the atomic operation includes, in response to determining that a current lock variable is in the unlocked state, exchanging an identification value corresponding to the any computing unit to the current lock variable and returning operation success, and in response to determining that the current lock variable is not in the unlocked state, returning operation failure; in response to the atomic operation performed with the current lock variable returning the operation success, determining that the any computing unit updates the current lock variable to the locked state, and in response to the atomic operation performed with the current lock variable returning the operation failure, determining that the any computing unit does not update the current lock variable to the locked state, and the any computing unit continues the atomic operation with a next lock variable until the R lock variables are looped through.

[0009] For example, in the data processing method provided by at least one of the embodiments of the present application, the atomic operation includes an atomic compare-and-swap operation, and the atomic compare-and-swap operation includes: comparing a value of the current lock variable with a preset value corresponding to the unlocked state, in response to the value of the current lock variable being the same as the preset value, determining that the current lock variable is in the unlocked state, and swapping an identification value corresponding to the any one of the plurality of computing units into the current lock variable, and returning the preset value to indicate that the operation is successful, in response to the value of the current lock variable being different from the preset value, determining that the current lock variable is not in the unlocked state, and returning the value of the current lock variable to indicate that the operation is unsuccessful, wherein the identification value corresponding to each of the plurality of computing units is different from the preset value.

[0010] For example, in the data processing method provided by at least one of the embodiments of the present application, initializing the contention workspace includes: allocating a global memory region on a memory of the parallel processor as the contention workspace; and setting the R lock variables in the contention workspace, wherein the R lock variables are initialized to the unlocked state.

[0011] For example, in the data processing method provided by at least one of the embodiments of the present application, determining at least one contention work block in the plurality of work blocks according to a number of work blocks included in input data of a target kernel function corresponding to the computing operation and a number of the plurality of computing units includes: determining a theoretical number W of at least one wave corresponding to the target kernel function according to a number N of the plurality of work blocks and a number M of the plurality of computing units, wherein each wave includes a parallel computing task of M work blocks performed by M computing units to complete one round of parallel task; determining a contention wave in the at least one wave according to the theoretical number W; and determining work blocks in the contention wave as the at least one contention work block, wherein M, N, and W are positive integers.

[0012] For example, in the data processing method provided by at least one of the embodiments of the present application, determining a contention wave in the at least one wave according to the theoretical number W includes: in response to W > 1, determining an mth executed wave and all waves after the mth executed wave in the W waves as the contention wave, m being a preset positive integer and being less than or equal to W, and in response to W = 1, determining the at least one wave as the contention wave.

[0013] For example, in the data processing method provided in at least one embodiment of this application, scheduling any one computing unit to execute a subsequent processor task includes: in response to the fact that the computing operation corresponding to the target kernel function is not the last computing operation executed among the plurality of computing operations, scheduling any one computing unit to immediately execute the computing task in the kernel function corresponding to the next computing operation.

[0014] For example, in the data processing method provided in at least one embodiment of this application, for the other work blocks among the plurality of work blocks except for the at least one competing work block, the allocation relationship between the plurality of computing units and the computing tasks of the other work blocks is determined according to a static preset mapping relationship.

[0015] For example, in the data processing method provided in at least one embodiment of this application, the computational processing operation and the communication processing operation with data dependency are executed in parallel to realize the fusion operator. The communication processing operation includes multiple data copy operations, and at least one of the multiple computational operations is executed in parallel with at least one data copy operation.

[0016] For example, in the data processing method provided in at least one embodiment of this application, the parallel processor is any one of a plurality of processors included in a multiprocessor system, the plurality of processors are configured to jointly execute a processing task, and each processor is configured to execute a part of the processing task, the processing task includes executing the general computing fusion operator, and the computational processing operation includes the portion of the general computing fusion operator executed by the parallel processor.

[0017] This invention provides at least one embodiment of a parallel processor, including multiple computing units. The parallel processor is configured to execute computational processing operations, wherein the computational processing operations include multiple computational operations executed sequentially. The input data of the computational processing operations is divided into multiple data blocks, and each computational operation is used to process one of the multiple data blocks. When the parallel processor executes each computational operation, it includes performing the following operations: determining at least one competing working block among the multiple working blocks based on the number of multiple working blocks included in the input data of the target kernel function corresponding to the computational operation and the number of the multiple computing units; and having the multiple computing units pipeline and parallelly process the computational tasks of the multiple working blocks to complete the computational operation. Specifically, for the at least one competing working block, a competition mechanism dynamically determines the allocation relationship between the multiple computing units and the computational tasks of the at least one competing working block. The competition mechanism includes the multiple computing units competing for the execution qualification of the computational task of the at least one competing working block, and the computing unit that obtains the execution qualification executes the computational task of the corresponding competing working block.

[0018] This application provides at least one embodiment of an electronic device, including a parallel processor as described in at least one embodiment of this disclosure.

[0019] This application provides at least one embodiment of an electronic device, including: a memory that non-transitoryly stores computer-executable instructions; and a processor configured to run the computer-executable instructions, wherein the computer-executable instructions are executed by the processor to implement the data processing method according to at least one embodiment of this disclosure.

[0020] This invention application discloses at least one embodiment of a non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the data processing method according to at least one embodiment of this disclosure.

[0021] The data processing method provided in this invention application introduces a competition wave and a dynamic competition mechanism when the competition wave takes effect. The competition mechanism dynamically determines the allocation relationship between the computing unit and the computing tasks corresponding to the work block, transforming the cumulative long-tail effect of multiple computing operations into a load problem that can be dynamically balanced, thereby eliminating the accumulated idle time and improving the overall performance. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0023] Figure 1 This is a schematic structural diagram of a general-purpose graphics processor;

[0024] Figure 2 This is a schematic diagram illustrating task allocation for a computing unit.

[0025] Figure 3A This is a schematic diagram of a converged computing scenario;

[0026] Figure 3B for Figure 3A The diagram shows the execution flow when performing matrix multiplication on the activation tensor and the weight tensor.

[0027] Figure 3C A schematic diagram illustrating task allocation for another type of computing unit;

[0028] Figure 4 A schematic flowchart illustrating a data processing method provided in at least one embodiment of this disclosure;

[0029] Figure 5This is a schematic diagram of task allocation for a computing unit provided in at least one embodiment of the present disclosure;

[0030] Figure 6 A schematic structural diagram of a parallel processor provided for at least one embodiment of this disclosure;

[0031] Figure 7 A schematic structural diagram of a multiprocessor system provided for at least one embodiment of the present disclosure;

[0032] Figure 8 A schematic structural diagram of an electronic device provided for at least one embodiment of this disclosure;

[0033] Figure 9 A schematic block diagram of another electronic device provided in one embodiment of this disclosure;

[0034] Figure 10 This is a schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0036] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0037] Figure 1 This is a schematic diagram of the structure of a general-purpose graphics processor.

[0038] like Figure 1 As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 1 The diagram shows multiple streaming processor clusters, all with identical structures. In a general-purpose graphics processor, one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0039] like Figure 1 As shown, a streaming processor cluster comprises multiple compute units (CUs), all with identical structures. Each CU performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A CU contains multiple cores, each including an arithmetic logic unit (ALU), a floating-point unit, etc., which execute specific computational tasks. Furthermore, a CU also includes registers (e.g.,...). Figure 1 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0040] like Figure 1 As shown, each computing unit also provides a tensor core for performing tensor-related computations, such as tensor shrinking operations. Tensor cores can accelerate tensor operations such as matrix multiplication. Tensor cores in multiple computing units can be scheduled and controlled uniformly.

[0041] like Figure 1 As shown, each streaming processor cluster also provides a buffer for data caching of multiple computing units within the streaming processor cluster.

[0042] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 1(Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0043] In each computing unit, the thread beam scheduling / distribution module ( Figure 1 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., [example cache]). Figure 1 High Bandwidth Memory (HBM) is used for read and write operations.

[0044] As mentioned above, the computing units of a graphics processor do not process work blocks one by one, but are scheduled in batches or waves. The GPU scheduler organizes computing tasks into multiple waves for execution, allowing as many computing units as possible to process tasks simultaneously in each wave.

[0045] In parallel processor programming (such as GPUs), a kernel is a piece of parallel code that runs on the device side and is initiated by the host (such as the central processing unit, CPU). It is the core logical unit for the parallel processor to execute computational tasks. The kernel defines the specific operations that each thread in the parallel processor needs to perform and is the key carrier for achieving data parallel computing. An operator may consist of one or more kernel functions, which are the carriers on the GPU where the operator is executed.

[0046] For a single kernel function, the number of working blocks in its input data is often not divisible by the number of computing units. For example, if there are 12 working blocks and 5 computing units, the parallel processor will process them in 3 waves: the first two waves each have 5 computing units processing 5 working blocks, the last wave has 2 computing units processing 2 working blocks, and the remaining 3 computing units are idle in this wave.

[0047] Therefore, in the above situation, the utilization rate of the computational units in the last wave is extremely low. This resource idleness at the end of kernel function execution due to load imbalance is called the long tail effect. It directly reduces the utilization rate of parallel processor hardware and prolongs the execution time of a single operator.

[0048] Figure 2 This is a schematic diagram of task allocation for a computing unit.

[0049] exist Figure 2 In the example, the input data of the kernel function is divided into 12 working blocks, which are: Figure 2 Work block 0, work block 1, work block 2, ..., work block 10, work block 11.

[0050] For example, in Figure 2 In the example, a statically fixed mapping method is used to allocate data blocks when launching the parallel processor to execute the kernel function. For example, computing unit 0 is fixed to process data blocks 0, 5, and 10; computing unit 1 is fixed to process data blocks 1, 6, and 11; computing unit 2 is fixed to process data blocks 2 and 7; computing unit 3 is fixed to process data blocks 3 and 8; and computing unit 4 is fixed to process data blocks 4 and 9. This simplifies the startup operation.

[0051] However, in this case, for a given input data, only two computing units are active in the third wave, while the other three computing units are idle, reducing the utilization of the graphics processor's hardware resources.

[0052] In distributed computing scenarios such as pre-training and post-training of large models, tensors can be partitioned using methods such as sequence parallelism and tensor parallelism. Sequence parallelism partitions the input sequence along the token dimension, with each processor processing only a portion of the tokens, reducing memory usage and computational load. Tensor parallelism partitions the model's weight tensor along its dimensions, resulting in multiple subtasks that process the different partitioned tensors. These subtasks are then distributed across different processors and executed in parallel.

[0053] Therefore, by using sequential parallelism or tensor parallelism, a computational task can be divided into multiple subtasks. Each subtask performs a portion of tensor computation operations, and multiple processors can execute multiple subtasks in parallel to complete the computational task, thereby improving throughput and shortening the completion time of the computational task.

[0054] Large models often involve scenarios where communication and computation overlap. Examples include the AllGather operator combined with the GEMM (Generative Geometric Multiplication Model) approach, or the matrix multiplication operator combined with the ReduceScatter operator. Currently, data communication and computation are performed serially in a pipeline.

[0055] Computation-communication integration is an optimization technique that enables the overlap of computation and communication. It breaks down a large computation kernel function into multiple smaller kernel functions and interleaves them with communication kernel functions to achieve the overlap of computation and communication, thereby improving the overall hardware utilization and program throughput.

[0056] Figure 3A This is a schematic diagram of a computing integration scenario.

[0057] like Figure 3A As shown, the activation tensor is divided into four parts, which are represented as follows: Figure 3A Tensors A0, A1, A2, and A3 are given, while the weight tensor is divided into four parts, represented as tensors W0, W1, W2, and W3 respectively. For example... Figure 3A As shown, matrix multiplication of tensor A0 and tensor W0 yields result D00, matrix multiplication of tensor A1 and tensor W1 yields result D11, and so on.

[0058] To minimize communication time within computation time, computation and communication operations are typically divided into multiple sub-operations and executed in parallel.

[0059] Figure 3B for Figure 3A The diagram shows the execution flow when performing matrix multiplication on the activation tensor and the weight tensor.

[0060] like Figure 3B As shown, parallel processor a includes a communication stream and a computation stream. The computation stream includes multiple matrix multiplication operations executed sequentially, namely W0×A0, W0×A1, W0×A2, and W0×A3. The communication stream of parallel processor a includes multiple data copy operations executed sequentially, namely copying tensor A0, copying tensor A1, copying tensor A2, and copying tensor A3.

[0061] Parallel processor b also includes a communication stream and a computation stream. The computation stream consists of multiple matrix multiplication operations executed sequentially, namely W1×A1, W1×A2, W1×A3, and W1×A0. The communication stream of parallel processor b consists of multiple data copy operations executed sequentially, namely copying tensor A1, copying tensor A2, copying tensor A3, and copying tensor A0.

[0062] The same applies to parallel processors c and d, which will not be elaborated here.

[0063] exist Figure 3B In this context, the horizontal direction represents the time axis, while overlapping matrix multiplication and data copying operations in the vertical direction are executed in parallel, such as the matrix multiplication operation W0×A0 and the operation of copying tensor A1.

[0064] Therefore, the matrix multiplication between the activation tensor and the weight tensor is divided into several smaller matrix multiplication operations, namely W0×A0, W0×A1, W0×A2, W0×A3, etc., and the communication operations and computation operations are executed in parallel as much as possible to improve the utilization of hardware resources and hide the time consumption caused by communication.

[0065] As mentioned above, in the context of integrated computing, a large kernel function is broken down into multiple consecutive smaller kernel functions, for example... Figure 3A The matrix multiplication between the activation tensor and the weight tensor executed by the parallel processor 0 shown is broken down into multiple consecutively executed matrix multiplication operations, namely W0×A0, W0×A1, W0×A2, and W0×A3, etc. Each matrix multiplication operation can be regarded as an independently executed kernel function.

[0066] Figure 3C A schematic diagram illustrating task allocation for another type of computing unit.

[0067] like Figure 3C As shown, for Figure 3A In the computational scenario shown, for parallel processor 0, four matrix multiplication operations are performed, namely W0×A0, W0×A1, W0×A2, and W0×A3, to complete the computational tasks of the portions of the activation tensor and weight tensor allocated to processor 0.

[0068] For each small kernel function (e.g., W0×A0), the input data is divided into, for example, 12 working blocks. Figure 3C In the diagram, A0_0-A0_11 represents the 12 working blocks of tensor A0, A1_0-A1_11 represents the 12 working blocks of tensor A1, A2_0-A2_11 represents the 12 working blocks of tensor A2, and A3_0-A3_11 represents the 12 working blocks of tensor A3.

[0069] In the case of static allocation, computation unit 0 is fixed to perform work block 0 for each input data (e.g., Figure 3C A0_0, A1_0, A2_0, A3_0 in the code), working block 5 (e.g. Figure 3C A0_5, A1_5, A2_5, A3_5 in the example), working block 10 (e.g. Figure 3CThe computation task for A0_10, A1_10, A2_10, and A3_10 in the dataset. Computation unit 2 performs the computation for each input data block 2 (e.g., ...). Figure 3C A0_2, A1_2, A2_2, A3_2 in the middle), working block 7 (e.g. Figure 3C The computation task of A0_7, A1_7, A2_7, and A3_7 in the dataset. Computation unit 4 performs the computation of each input data work block 4 (e.g., ...). Figure 3C A0_4, A1_4, A2_4, A3_4 in the middle), working block 9 (e.g. Figure 3C The computational tasks for A0_9, A1_9, A2_9, and A3_9 in the dataset are as follows.

[0070] like Figure 3C As shown, Figure 3A The computation of the activation and weight tensors, as shown, generates a long-tail effect independently within each small-scale kernel function (one matrix multiplication operation). These long-tail effects accumulate, causing the total idle time to increase exponentially. This is because in parallel computing, the task load cannot be evenly distributed, resulting in most computational units being idle at the end of the task execution, with only a few units working, leading to overall resource waste. The end result is that the cumulative execution time of multiple small kernel functions significantly exceeds the execution time of a single large kernel function. This contradicts the initial goal of performance improvement in parallel computing and becomes a serious performance bottleneck in this technological path.

[0071] This disclosure provides at least one embodiment of a data processing method applied to a parallel processor. The parallel processor includes multiple computing units. The data processing method includes: executing computational processing operations through the parallel processor, wherein the computational processing operations include multiple computational operations executed sequentially, the input data of the computational processing operations are divided into multiple data blocks, and each computational operation is used to process one of the multiple data blocks. Executing the multiple computational operations through the parallel processor includes: for each computational operation: determining at least one competing working block among the multiple working blocks based on the number of multiple working blocks included in the input data of the target kernel function corresponding to the computational operation and the number of the multiple computing units; and having the multiple computing units pipeline and parallelly process the computational tasks of the multiple working blocks to complete the computational operation. For the at least one competing working block, a competition mechanism dynamically determines the allocation relationship between the multiple computing units and the computational tasks of the at least one competing working block. The competition mechanism includes the multiple computing units competing for the execution qualification of the computational task of the at least one competing working block, and the computing unit that obtains the execution qualification executes the computational task of the corresponding competing working block.

[0072] The data processing method provided in at least one embodiment of this disclosure introduces a competing work block and applies a dynamic competition mechanism to the competing work block. The competition mechanism dynamically determines the allocation relationship between the computing unit and the computing tasks of the competing work block, transforming the cumulative long-tail effect of multiple computing operations included in the computing processing operation into a load problem that can be dynamically balanced, thereby eliminating the accumulated idle time and improving the overall performance.

[0073] Specifically, the data processing method provided in at least one embodiment of this disclosure no longer passively accepts the uneven load at the end of each kernel function, but actively performs "peak shaving and valley filling." Instead of pre-assigning a few computing units to handle the remaining work blocks in each kernel function, multiple computing units compete for these workloads through a fair competition mechanism. In this way, a computing unit idle in the first kernel function may obtain a competing work block in the second kernel function; and a unit idle in the second kernel function may obtain a competing work block in the third kernel function. From a macroscopic perspective, the workload of all computing units becomes more balanced throughout the task, compressing the overall idle time, thereby eliminating the cumulative long-tail effect and improving hardware utilization and execution efficiency.

[0074] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0075] The data processing method provided in at least one embodiment of this disclosure can be applied to parallel processors, including graphics processors, general-purpose graphics processors, tensor processing units (TPUs), data processing units (DPUs), neural processing units (NPUs), and other processors capable of parallel data processing. This disclosure does not impose specific limitations on these processors.

[0076] For example, in some embodiments, the parallel processor may be a graphics processor or a general-purpose graphics processor, and its hardware architecture may refer to Figure 1 Related descriptions.

[0077] For example, a parallel processor includes multiple computing units that can execute the same or different computing tasks in parallel, such as performing the same processing on different input data.

[0078] The data processing method provided in at least one embodiment of this disclosure includes performing computational processing operations using a parallel processor.

[0079] For example, a computational processing operation includes multiple computational operations executed sequentially. The input data of the computational processing operation is divided into multiple data blocks, and each computational operation is used to process one of the multiple data blocks.

[0080] The computational processing operation can be any computational operation that a parallel processor needs to execute, such as matrix multiplication and accumulation operations, and this disclosure does not impose any specific restrictions on it.

[0081] For example, in the aforementioned Figure 3A and Figure 3B In the example, the computational processing operation can be a matrix multiplication operation executed by a parallel processor a. Multiple computational operations can be matrix multiplication operations A0×W0, A1×W0, A2×W0, and A3×W0, and multiple data blocks can be tensors A0, A1, A2, and A3. Of course, this disclosure is not limited to this; the computational processing operation, computational operations, and data blocks can differ for different operators, and this disclosure does not impose specific limitations in this regard.

[0082] For example, in one example, computational processing operations are executed in parallel with communication processing operations that have data dependencies to implement a computation-computation fusion operator. For example, the computation-computation fusion operator can be an operator in which communication and computation are executed overlappingly, such as the AllGather operator + matrix multiplication operator (GEMM), or the matrix multiplication operator + reduce scattering operator, etc. This disclosure does not impose specific limitations on the computation-computation fusion operator.

[0083] The communication processing operation includes multiple data copy operations, and at least one of the multiple computation operations is executed in parallel with at least one data copy operation.

[0084] For example, refer to Figure 3B For example, the communication processing operation includes copying tensor A0, copying tensor A1, copying tensor A2, and copying tensor A3. During the execution cycle of the computation operation A0×W0, the copying of tensor A1 is also executed in parallel; during the execution cycle of the computation operation A1×W0, the copying of tensor A2 is also executed in parallel, and so on. The computational fusion operator breaks down what would otherwise be a large computational processing operation into multiple smaller computational operations, which are interspersed with data copying operations to achieve overlap between computation and communication, thereby improving overall hardware utilization and program throughput.

[0085] It should be noted that this disclosure is not limited thereto, and this disclosure does not impose specific restrictions on the parallel execution of communication processing operations and computation processing operations.

[0086] For example, in one example, the parallel processor is any one of a plurality of processors included in a multiprocessor system, the plurality of processors being configured to jointly execute a processing task, and each processor being configured to perform a portion of the processing task, the processing task including the execution of a general computation fusion operator, the computational processing operation including a portion of the processing task performed by the parallel processor in the general computation fusion operator.

[0087] For example, in Figure 3B In the example, the parallel processor can be Figure 3B Any one of the parallel processors a, b, c, and d in the configuration. The four parallel processors are configured to work together to complete... Figure 3A The matrix multiplication of the weight tensor and activation tensor in the computation process can include four matrix multiplication operations executed by a parallel processor.

[0088] For example, multiple processors can be graphics processors, general-purpose graphics processors, etc.

[0089] For example, the multiprocessor system can be a multi-GPU system, which integrates multiple GPU cards and works together to complete large-scale parallel computing tasks. It is commonly used for tasks such as artificial intelligence training, scientific computing, and high-performance graphics rendering.

[0090] Figure 4 This is a schematic flowchart illustrating a data processing method provided in at least one embodiment of the present disclosure.

[0091] For example, performing computational processing operations using a parallel processor includes performing multiple computational operations using a parallel processor, where each computational operation may include...Figure 4 Steps S10 and S20 in the process.

[0092] In step S10, based on the number of multiple working blocks and the number of multiple computing units included in the input data of the target kernel function corresponding to the computation operation, at least one competing working block among the multiple working blocks is determined.

[0093] For example, with Figure 3A For example, when the parallel processor is parallel processor a, the target kernel function can be kernel1 corresponding to the computation operation A0×W0, kernel2 corresponding to the computation operation A1×W0, kernel3 corresponding to the computation operation A2×W0, or kernel4 corresponding to the computation operation A3×W0. The same principle applies to other parallel processors, and will not be elaborated further here.

[0094] For example, taking the target kernel function used to perform the computation operation A0×W0 as an example, the input data are A0 and W0. The input data can be divided into multiple working blocks, and the tensors A0 and W0 can also be divided into multiple working blocks, for example... Figure 3C For example, tensor A0 can be divided into 12 working blocks, including tensors A0_0 to A0_11. Similarly, tensor W0 can be divided into 12 working blocks of the same size. Each computational unit executes the computational task of one working block, for example, executing the computational tasks of the corresponding working blocks in tensors A0_0 and W0. The same principle applies when the target kernel function is used for other computational operations, which will not be elaborated further here.

[0095] The number of computing units can be the total number of computing units included in a parallel processor for performing computational operations.

[0096] For example, in some embodiments, step S10 may include: determining the theoretical number W of at least one wave corresponding to the target kernel function based on the number N of the plurality of working blocks and the number M of the plurality of computing units, wherein each wave includes the computational task of M working blocks performed in parallel by M computing units to complete one round of parallel task; determining the competing wave among the at least one wave based on the theoretical number W; and determining the working blocks in the competing wave as the at least one competing working block, wherein M, N, and W are all positive integers.

[0097] The input data of the target kernel function is divided into multiple working blocks. Each computing unit executes a computing task for one working block at a time. Each wave consists of multiple computing units performing multiple computing tasks in parallel to complete one round of parallel tasks.

[0098] In parallel processors such as graphics processing units (GPUs), a wave is essentially a set of threads that execute the same instructions synchronously. It is a logical grouping of computing units in a parallel processor to execute tasks. Computing units within a wave usually start synchronously and execute the same or different tasks.

[0099] In one wave, M computing units perform M computing tasks in parallel to complete one round of parallel tasks. For example, a parallel processor includes 5 computing units for performing computing operations. In one wave, the 5 computing units perform the computing tasks of 5 working blocks in parallel. For example, in the first wave, the 5 computing units perform the computing tasks of working blocks A0_0, A0_1, A0_2, A0_3, and A0_4 in parallel.

[0100] It should be noted that the at least one wave corresponding to the target kernel function refers to the theoretical number of waves obtained when multiple computational units are used only to execute the target kernel function. For example, the theoretical number W of at least one wave corresponding to the target kernel function can be determined according to the following formula: W = ceil(N / M). Where ceil() is the floor function, N represents the number of multiple working blocks, and M represents the number of multiple computational units.

[0101] For example, refer to Figure 3C The kernel function kernel1 corresponds to at least one wave, which can be the 1st to the 3rd wave (columns 1 to 3 from the left). The kernel function kernel2 corresponds to at least one wave, which can be the 4th to the 6th wave (columns 4 to 6 from the left). The kernel function kernel3 corresponds to at least one wave, which can be the 7th to the 9th wave (columns 7 to 9 from the left). The kernel function kernel4 corresponds to at least one wave, which can be the 10th to the 12th wave (columns 10 to 12 from the left).

[0102] When a parallel processor actually executes a computational operation, the first computational operation is actually executed at least one wave corresponding to the target kernel function. However, for subsequent computational operations, since some computational tasks can be performed earlier, the waves actually executed by the parallel processor may not be exactly the same as the at least one wave corresponding to the target kernel function.

[0103] The at least one wave is executed sequentially in a pipelined manner, and therefore has a predetermined execution order. Even if some computational tasks start earlier during actual execution, the relative order within the kernel function still maintains the predetermined execution order.

[0104] For example, in some embodiments, determining the competing wave among the at least one wave based on the theoretical number of the at least one wave may include: in response to W > 1, determining the m-th wave executed among W waves and all waves after the m-th wave executed as the competing wave, where m is a preset positive integer less than or equal to W; and in response to W = 1, determining the 1 wave as the competing wave.

[0105] For example, in response to W > 1, the m-th wave executed out of W waves is determined as the first wave. The competing waves include the first wave and all waves executed after the first wave (if any). That is, a certain wave and all waves after it (if any) can be regarded as competing waves.

[0106] m is a preset value. Different kernel functions for computational processing operations can use the same m. For example, if m=W, then only the last wave is considered as the competing wave; or if m=2, then the second wave and subsequent waves are all considered as competing waves. The setting of m can be set according to actual needs, and this disclosure does not impose specific restrictions on it.

[0107] For example, in Figure 3C In the example shown, N=12, M=5, W=ceil(12 / 5)=3, m=W=3. Since W>1, the third wave is taken as the competing wave. The working blocks in the competing wave are A0_10 and A0_11. Working blocks A0_10 and A0_11 are taken as competing working blocks.

[0108] If W=1, then there is only one wave, which is directly used as the competing wave, meaning all working blocks are used as competing working blocks.

[0109] For example, in other embodiments, competing work blocks can be specified, such as work blocks A0_10 and A0_11 as competing work blocks.

[0110] For example, in some embodiments, only the last wave of at least one wave is designated as the competing wave, and the working blocks within the competing wave are designated as competing working blocks. In this embodiment, the overhead of the competition mechanism and atomic operations only occurs in the last wave with the least computational load; that is, only the last wave is designated as the competing wave. For the other waves that constitute the majority, their execution is no different from traditional methods, without any additional overhead. This design ensures that while addressing core performance bottlenecks, it does not have a significant negative impact on overall computational performance, performance overhead is controllable, and fine-grained optimization is achieved.

[0111] In step S20, multiple computing units process the computing tasks of multiple work blocks in a pipelined parallel manner to complete the computing operation.

[0112] For the at least one competing work block, a competition mechanism is used to dynamically determine the allocation relationship between multiple computing units and the computing tasks of the at least one competing work block. The competition mechanism includes the competition among the multiple computing units for the execution qualification of the computing tasks of the at least one competing work block. The computing unit that obtains the execution qualification executes the computing tasks of the corresponding competing work block. Each computing unit executes the computing tasks of one competing work block at a time.

[0113] Therefore, unlike the pre-specified approach for competing work blocks, this method allows all computing units to compete for these workloads through a fair competition mechanism. This transforms the cumulative long-tail effect caused by multiple small-scale kernel functions in operators such as general computing fusion into a dynamically balanced load problem, thereby eliminating accumulated idle time and improving overall performance.

[0114] The other work blocks (e.g., non-contested work blocks) among multiple work blocks are still assigned according to the static preset mapping relationship, which determines the allocation relationship between the computing unit and the computing task corresponding to the work block.

[0115] For example, such as Figure 3C As shown, non-contested work blocks can be work blocks A0_0-A0_9, A1_0-A1_9, A2_0-A2_9, and A3_0-A3_9, while contested work blocks can be work blocks A0_10-A0_11, A1_10-A1_11, A2_10-A2_11, and A3_10-A3_11.

[0116] For example, for these working blocks located in normal wavelets, refer to Figure 3C For example, each computing unit performs the computing task corresponding to a fixed work block. For instance, computing unit 3 performs the computing task for each input data work block 3 (e.g., ...). Figure 3C A0_3, A1_3, A2_3, A3_3 in the example), working block 8 (e.g. Figure 3C The computational task of A0_8, A1_8, A2_8, A3_8 in the dataset.

[0117] For example, at least one wave corresponding to the target kernel function includes a competing wave, and the working blocks in the competing wave serve as the at least one competing working block. For any one of the plurality of computing units, after the computing unit completes the computation task of the target working block, atomic operations are used cyclically to compete for the execution qualification of the at least one competing working block, wherein the target working block includes working blocks located in the previous wave that is before and adjacent to the first competing wave; in response to any computing unit obtaining the execution qualification of any competing working block, the computing unit executes the computation task of the competing working block; in response to any computing unit not obtaining the execution qualification of any competing working block, the computing unit is scheduled to execute subsequent processor tasks.

[0118] For example, in response to the number of the at least one wave being greater than 1, the previous wave belongs to the at least one wave; in response to the number of the at least one wave being equal to 1, the previous wave does not belong to the at least one wave, for example, it belongs to a wave in the previous computational operation or a wave in the previous computational processing operation.

[0119] For example, in one example, still taking a work block consisting of A0_0-A0_11 and a computation unit count of 5, work blocks A0_10 and A0_11 in the third wave are competing work blocks, while the other work blocks A0_0-A0_9 are non-competitive work blocks. Since the number of waves is greater than 1, each computation unit still allocates the non-competitive work blocks in the first two waves according to a static preset mapping relationship. For example, computation unit 3 is fixed to process work block 3 for each input data (e.g., ...). Figure 3C A0_3, A1_3, A2_3, A3_3 in the example), working block 8 (e.g. Figure 3C The computational task of A0_8, A1_8, A2_8, A3_8 in the dataset.

[0120] The competition wave is the third wave, and the working blocks in the previous wave are working blocks 5-9.

[0121] For computation unit 0, the target working block can be working block 5 for each input data (e.g., A0_5, A1_5, etc.). For computation unit 1, the target working block can be working block 6 for each input data (e.g., A0_6, A1_6, etc.). For computation unit 2, the target working block can be working block 7 for each input data (e.g., A0_7, A1_7, etc.). For computation unit 3, the target working block can be working block 8 for each input data (e.g., A0_8, A1_8, etc.). For computation unit 4, the target working block can be working block 9 for each input data (e.g., A0_9, A1_9, etc.).

[0122] After the computing unit completes the execution of the target work block, atomic operations are used cyclically to compete for the execution qualification of the at least one competing work block.

[0123] For example, if a computing unit obtains the qualification to execute any competing work block, the computing unit executes the computing task of that competing work block; if a computing unit does not obtain the qualification to execute any competing work block, the computing unit is scheduled to execute the subsequent processor tasks.

[0124] For example, in some embodiments, scheduling any one of the computing units to execute subsequent processor tasks may include: in response to the fact that the computing operation corresponding to the target kernel function is not the last computing operation executed among the plurality of computing operations, scheduling any one of the computing units to immediately execute the computing task in the kernel function corresponding to the next computing operation.

[0125] In other words, subsequent processor tasks can be computational tasks within the kernel function corresponding to the next computational operation. For example, this computational task can be a task determined based on a static, pre-defined mapping relationship.

[0126] The computational tasks in the kernel function corresponding to the next computational operation are performed in parallel with the corresponding computational tasks executed by the computational unit that has won the right to compete for the work block.

[0127] In this embodiment, since the kernel functions corresponding to the computation operations are independent of each other, the computation task of the next kernel function can start in advance without waiting for the current kernel function to finish executing, thus reducing the overall idle waiting time.

[0128] If the computation operation corresponding to the target kernel function is the last computation operation executed among the multiple computation operations, then after all the computation tasks of the competing work block have been executed, the scheduling computation unit synchronously starts the task of the next computation processing operation.

[0129] For example, in some embodiments, dynamically determining the allocation relationship between computing units and the computing tasks of the at least one competing work block through a competition mechanism may include: initializing a competition workspace, wherein the competition workspace includes R lock variables initialized to an unlocked state, where R is a positive integer representing the number of the at least one competing work block, and the R lock variables correspond one-to-one with the at least one competing work block; multiple computing units cyclically using atomic operations to compete for the R lock variables in the competition workspace to determine whether any lock variable can be updated from the unlocked state to the locked state; determining that a first computing unit that updates a lock variable to the locked state obtains the qualification to execute the computing task of the competing work block corresponding to the lock variable, the first computing unit withdraws from the competition, and executes the computing task of the competing work block corresponding to the lock variable; determining that a second computing unit that has not updated any lock variable to the locked state ends the execution operation of the target kernel function, and scheduling the second computing unit to execute subsequent processor tasks.

[0130] For example, if the last wave is a competing wave, R = N - M × (W - 1).

[0131] Lock variables are a basic synchronization mechanism used in concurrent programming to achieve mutual exclusion. Their core purpose is to ensure that multiple threads / processes can access shared resources mutually exclusively (i.e., only one thread / process can enter the critical section at a time), thus avoiding data races or inconsistency problems.

[0132] By binding lock variables to unprocessed work blocks, access consistency is guaranteed, ensuring that only one computational unit is eligible to execute a work block.

[0133] For example, in some embodiments, initializing the contention workspace may include: allocating a global memory region on the memory of the parallel processor as the contention workspace; and setting R lock variables in the contention workspace, wherein the R lock variables are initialized to an unlocked state.

[0134] For example, in Figure 1 The graphics processor shown pre-allocates a global memory region on its high-bandwidth memory as the competition workspace corresponding to the competition wave. This region contains R lock variables, which are initialized to an unlocked state before the target kernel function is executed, for example, initialized to 0.

[0135] All currently available computing units compete for a lock in the competing workspace, with each unit attempting to acquire one through a loop using atomic operations. Here, "currently available computing units" refers to those that are idle during the current wave of execution by the parallel processor.

[0136] If a computing unit can update a lock variable in the competing workspace from an unlocked state to a locked state, it means that the computing unit has successfully won the lock and has been granted the right to execute the computing task of the work block corresponding to the lock variable. The computing unit then exits the competition loop and begins to execute the computing task of the work block it has won.

[0137] If a computation unit fails to acquire any of the R lock variables after traversing them (i.e., cannot update any lock scalar from an unlocked state to a locked state), it indicates that all remaining work in the current contention wave has been claimed / allocated by other computation units. The computation unit can then directly terminate the execution of the current target kernel function and can be scheduled to execute subsequent processor tasks, such as the computation task of the next kernel function.

[0138] Atomic operations are a core concept in concurrent programming and computer architecture. They refer to the smallest, uninterruptible unit of operation, whose execution cannot be interrupted by other threads / processes or hardware. They either complete completely or not at all, with no intermediate state. Atomic operations ensure that only one computational unit can modify a lock variable at a time. Other computational units can only execute operations on the lock variable after that unit has finished its operation, guaranteeing access consistency and ensuring that only one computational unit is eligible to execute a work block.

[0139] For example, the atomic operation used by the computing unit can be one or a combination of multiple atomic operations, as long as it can achieve the function of the atomic operation defined in this disclosure. This disclosure does not impose specific restrictions on the specific atomic operation.

[0140] In this embodiment, the allocation relationship between computing units and computing tasks corresponding to competing work blocks is dynamically determined through a competition mechanism, eliminating the cumulative long-tail effect caused by static allocation that splits computing operations into multiple computing operations, thereby improving overall performance. Through the dynamic competition mechanism, the scattered and fixed long-tail loads in multiple kernel functions are redistributed, making the load of all computing units more balanced in the entire fusion task, which fundamentally solves the problem of cumulative long-tail effect.

[0141] As mentioned earlier, to achieve parallel communication and computational processing operations, the general-purpose computing fusion operator executes multiple computational and data copy operations in parallel. The resulting long-tail effect leads to a significant increase in idle time, substantially reducing the performance of the general-purpose computing fusion operator. The embodiments disclosed in this disclosure can eliminate the cumulative long-tail effect, allowing the total execution time of multiple computational operations to approach or even reach the performance of a single computational processing operation, ensuring the effectiveness of the general-purpose computing fusion technology and improving overall hardware utilization and program throughput.

[0142] For example, in some embodiments, multiple computing units cyclically use atomic operations to compete for the R lock variables in the competition workspace to determine whether any lock variable can be updated from the unlocked state to the locked state. This can include: for any one of the multiple computing units: cyclically traversing the R lock variables, performing the atomic operation with each of the R lock variables, wherein the atomic operation includes, in response to determining that the current lock variable is in the unlocked state, swapping the identifier value corresponding to the computing unit with the current lock variable and returning operation success; in response to determining that the current lock variable is not in the unlocked state, returning operation failure; in response to the atomic operation performed with the current lock variable returning operation success, determining that the computing unit has updated the current lock variable to the locked state; in response to the atomic operation performed with the current lock variable returning operation failure, determining that the computing unit has not updated the current lock variable to the locked state, the computing unit continues to perform the atomic operation with the next lock variable until all R lock variables have been traversed.

[0143] For example, in some embodiments, the atomic operation includes an atomic comparison and swap operation, which includes: comparing the value of the current lock variable with a preset value corresponding to the unlocked state; in response to the current lock variable being the same as the preset value, determining that the current lock variable is in an unlocked state, swapping the identifier value corresponding to any one of the computation units to the current lock variable, and returning the preset value to indicate that the operation was successful; in response to the current lock variable being different from the preset value, determining that the current lock variable is not in an unlocked state, and returning the value of the current lock variable to indicate that the operation failed, wherein the identifier value corresponding to each of the multiple computation units is different from the preset value.

[0144] The identifier values ​​corresponding to multiple computing units can be the same or different from each other.

[0145] For example, the atomic compare-and-swap operation can be implemented using `atomicCAS(&workspace_lock[j], value_unlock, my_unique_id)`, where `workspace_lock[j]` represents the j-th lock variable out of R lock variables, where j ranges from 0 to R-1, `my_unique_id` is the identifier value corresponding to the computation unit, and `value_unlock` represents the preset value corresponding to the unlocked state, such as 0. `atomicCAS` represents an indivisible atomic operation. Specifically, it reads the current value of `workspace_lock[j]`; compares the current value with `value_unlock`; if they are equal, it writes `my_unique_id` to `workspace_lock[j]` and returns `value_unlock`; otherwise, it directly returns the current value in `workspace_lock[j]`.

[0146] The R lock variables workspace_lock[0]-workspace_lock[R-1] correspond one-to-one with the R competing work blocks.

[0147] For example, each of the currently available computing units iterates through R lock variables (workspace_lock[0] to workspace_lock[R-1]). For example, for the i-th computing unit (e.g., i ranges from 0 to M-1), when it performs an atomic comparison and swap operation on workspace_lock[j], such as atomicCAS (&workspace_lock[j], value_unlock, my_unique_id), if it returns value_unlock, it means that the operation was successful, and the index value my_unique_id corresponding to the i-th computing unit is also returned. Once written into workspace_lock[j], the i-th computation unit successfully updates the lock variable workspace_lock[j] to a locked state, confirming that the i-th computation unit has obtained the qualification to execute the computation task of the competing work block corresponding to workspace_lock[j]. The i-th computation unit exits the competition and executes the computation task of the competing work block corresponding to workspace_lock[j]. If the returned value is not value_unlock, it means that the operation has failed. Then, it continues to try to perform an atomic comparison and swap operation with the next lock variable until all R lock variables have been traversed.

[0148] If the i-th computation unit fails to update any of the R lock variables to the locked state after traversing them (for example, each atomic compare and swap operation does not return value_unlock), it indicates that the computation tasks of all remaining competing work blocks have been assigned to other computation units. In this case, the i-th computation unit is determined to end the execution of the target kernel function and is scheduled to execute subsequent processor tasks, such as continuing to the next kernel function.

[0149] It should be noted that this disclosure is not limited to using the atomic comparison and exchange operation atomicCAS to implement atomic operations. Those skilled in the art can also use other atomic operations or combinations of multiple atomic operations to implement the above process, and this disclosure does not impose any specific restrictions on this.

[0150] The following is based on Figure 3A and Figure 3B Taking the scenario shown as an example, the specific process of applying the data processing method provided by at least one embodiment of this disclosure is described.

[0151] Figure 5 This is a schematic diagram of task allocation for a computing unit provided in at least one embodiment of the present disclosure.

[0152] For example, taking parallel processor a as an example, the specific process of the computational processing operation performed by parallel processor a is described. The same applies to parallel processors b to d, which will not be repeated here.

[0153] refer to Figure 3B The computational processing operations executed by parallel processor a include four sequentially executed computational operations: A0×W0, A1×W0, A2×W0, and A3×W0. Figure 5 The diagram illustrates the task allocation for the four computational operations, and uses thick black lines to separate the ranges of the four kernel functions.

[0154] Figure 5 Work blocks with medium gray shading are competing work blocks, while work blocks with white backgrounds are non-competing work blocks.

[0155] First, the kernel function corresponding to A0×W0 is calculated.

[0156] Specifically, referring to step S10, based on the number of multiple working blocks and the number of multiple computing units included in the input data of the kernel function kernel1, at least one competing working block among the multiple working blocks is determined.

[0157] As mentioned earlier, tensor A0 is divided into 12 working blocks, and the number of computational units is 5. Therefore, the theoretical number of waves corresponding to at least one kernel function kernel1 used to execute A0×W0 is 3. Taking the last wave as the competing wave, refer to...Figure 3C The competing work blocks include work blocks A0_10 and A0_11 in the third wave.

[0158] Other non-contested work blocks include work blocks A0_0-A0_9. Non-contested work blocks are assigned tasks according to a static preset mapping relationship.

[0159] like Figure 5 As shown, in the first and second waves, the computational tasks of the work blocks are allocated to the computational units according to a static preset mapping relationship. In these two waves, each of the five computational units is assigned a work block according to a preset mapping rule and performs computations in parallel. For example, as... Figure 5 As shown, in the first two waves (such as the first and second columns from the left in 5), calculation unit 0 performs the calculation tasks of working blocks A0_0 and A0_5, calculation unit 1 performs the calculation tasks of working blocks A0_1 and A0_6, calculation unit 2 performs the calculation tasks of working blocks A0_2 and A0_7, and so on.

[0160] In the third wave, the contention wave, kernel1 still has 12 - 5 × 2 = 2 working blocks yet to be processed. The initialized contention workspace includes two lock variables, each corresponding to one of the two working blocks A0_10 and A0_11. The two lock variables are initialized to 0 to indicate an unlocked state. Each of the five computation units has a unique corresponding identifier value; for example, computation units 0 to 4 correspond to identifier values ​​1 to 5, respectively.

[0161] Each of the five computation units uses atomic operations in a loop to compete for two lock variables in an attempt to update either lock variable from an unlocked state to a locked state.

[0162] For example, computation unit 0 iterates through two lock variables and performs atomic comparison and swap operations on the lock variables, such as atomicCAS. For details, please refer to the aforementioned embodiment; further explanation is omitted here. If the atomic comparison and swap operation with the lock variable corresponding to work block A0_10 returns 0, it indicates that the operation was successful. Computation unit 0 has written the identifier value 1 to the lock variable corresponding to work block A0_10, confirming that computation unit 0 has obtained the qualification to execute the computation task of work block A0_10. Computation unit 0 then exits the competition and executes the computation task of work block A0_10.

[0163] At this time, the value of the lock variable corresponding to work block A0_10 is 1, which is a locked state. When other computing units perform atomic comparison and swap operations, the value of the lock variable corresponding to work block A0_10 is different from 0, and all return operation failure.

[0164] For example, after computation unit 1 performs an atomic comparison and swap operation with the lock variable corresponding to work block A0_11, it returns an operation failure. Computation unit 1 then performs another atomic comparison and swap operation with the lock variable corresponding to work block A0_11. For example, if it returns 0, it means that the operation was successful. Computation unit 1 writes the identifier value 2 into the lock variable corresponding to work block A0_11, confirming that computation unit 1 has obtained the qualification to execute the computation task of work block A0_11. Computation unit 1 withdraws from the competition and executes the computation task of work block A0_11.

[0165] At this time, the value of the lock variable corresponding to work block A0_11 is 2, which is a locked state. When other computing units perform atomic comparison and swap operations, the value of the lock variable corresponding to work block A0_11 is different from 0, and all return operation failure.

[0166] After traversing two lock variables, computation units 2-4 failed to update either lock variable to a locked state. Therefore, computation units 2-4 ended the execution of the target kernel function kernel1 and scheduled computation units 2-4 to immediately execute subsequent processor tasks, such as executing the operation of the next kernel function kernel2.

[0167] like Figure 5 As shown, in the third wave of execution by the parallel processor, computing units 0 and 1 obtain the execution qualification for work blocks A0_10 and A0_11. After computing units 0 and 1 complete the computation tasks of work blocks A0_10 and A0_11, they determine to end the execution operation for the target kernel function kernel1 and can schedule computing units 0 and 1 to execute subsequent processor tasks.

[0168] like Figure 5 As shown, in the third wave of execution by the parallel processor, computation units 2-4 did not qualify to execute the remaining working blocks in kernel1. Therefore, computation units 2-4 terminated the execution of the target kernel function kernel1 and could directly begin executing the computation tasks of working blocks A1_2-A1_4 in kernel function kernel2. The computation tasks executed by computation units 2-4 can be determined according to a static preset mapping relationship.

[0169] Next, the execution process of the kernel function kernel2 corresponding to the computation operation A1×W0 is described.

[0170] Specifically, referring to step S10, at least one competing working block is determined from the multiple working blocks based on the number of multiple working blocks and the number of multiple computing units included in the input data of the target kernel function kernel1.

[0171] As mentioned earlier, tensor A1 is divided into 12 working blocks, and the number of computational units is 5. Therefore, the theoretical number of waves corresponding to at least one kernel function kernel2 used to execute A1×W0 is 3. Taking the last wave as the competing wave, refer to... Figure 3C The competing working blocks include working blocks A1_10 and A1_11 in the third wave.

[0172] Other non-contested work blocks include work blocks A0_1-A1_9. Tasks are assigned to non-contested work blocks according to a static preset mapping relationship.

[0173] like Figure 5 As shown, in the third wave of execution by the parallel processor, computing units 2-4, having not obtained any qualification to execute any competing working blocks, directly begin executing working blocks A1_2-A1_4 of kernel2. In the fourth wave of execution by the parallel processor, computing unit 0 executes the computation task of working block A1_0 according to the preset mapping relationship, computing unit 2 executes the computation task of working block A1_7, computing unit 4 executes the computation task of working block A1_9, and so on.

[0174] In the fifth wave actually executed by the parallel processor, computing unit 0 executes non-contested work block A1_5 according to the preset mapping relationship, and computing unit 1 executes non-contested work block A1_6 according to the preset mapping relationship.

[0175] exist Figure 5 In the fifth wave, for non-contested work blocks A1_10 and A1_11, the initialized contested workspace includes two lock variables that correspond one-to-one with the two work blocks A1_10 and A1_11. The lock variables are initialized to 0 to indicate an unlocked state.

[0176] After computation units 2-4 have completed their respective target work blocks, each of the three computation units uses atomic operations in a loop to compete for two lock variables in an attempt to update either lock variable to a locked state.

[0177] For example, computation unit 2 iterates through the two lock variables and performs atomic comparison and swap operations on the lock variables, such as atomicCAS. For details, please refer to the aforementioned embodiment; further explanation is omitted here. If the atomic comparison and swap operation with the lock variable corresponding to work block A1_10 returns 0, it indicates that the operation was successful. Computation unit 2 has written the identifier value 3 into the lock variable corresponding to work block A1_10, confirming that computation unit 2 has obtained the qualification to execute the computation task of work block A1_10. Computation unit 2 then exits the competition and executes the computation task of work block A1_10.

[0178] At this time, the value of the lock variable corresponding to work block A1_10 is 3, which is a locked state. When other computing units perform atomic comparison and swap operations, the value of the lock variable corresponding to work block A1_10 is different from 0, and all return operation failure.

[0179] For example, after performing an atomic comparison and swap operation with the lock variable corresponding to work block A1_10, calculation unit 3 returns an operation failure. Then, calculation unit 3 performs an atomic comparison and swap operation with the lock variable corresponding to work block A1_11. For example, if it returns 0, it means that the operation is successful. Calculation unit 3 writes the identifier value 4 into the lock variable corresponding to work block A1_11 to determine that calculation unit 3 has obtained the qualification to execute the calculation task of work block A1_11. Calculation unit 3 withdraws from the competition and executes the calculation task of work block A1_11.

[0180] At this time, the value of the lock variable corresponding to work block A1_11 is 4, which is a locked state. When other computing units perform atomic comparison and swap operations, the value of the lock variable corresponding to work block A1_11 is different from 0, and all return operation failure.

[0181] After traversing the two lock variables, computation unit 4 did not update either lock variable to a locked state. Therefore, computation unit 4 terminates the execution of the target kernel function kernel2 and schedules computation unit 4 to execute subsequent processor tasks, such as directly starting the execution of the next kernel function kernel3. Figure 5 In the fifth wave, computing unit 4 executes the computation task of work block A2_4. The computation task executed by computing unit 4 at this time is also determined according to the static preset mapping relationship.

[0182] like Figure 5 As shown, in the 5th wave, computing units 2 and 3 obtain the execution qualification of work blocks A1_10 and A1_11. After computing units 2 and 3 finish executing the computing tasks of work blocks A1_10 and A1_11, they determine to end the execution operation on the target kernel function kernel2 and can schedule computing units 2 and 3 to execute subsequent processor tasks.

[0183] After computation units 0 and 1 finish executing the target working block, for example, after computation unit 0 finishes executing working block A1_5 and computation unit 1 finishes executing A1_6, computation units 0 and 1 also perform the above-mentioned operation of traversing the two lock variables. Since both lock variables have been updated to the locked state, computation units 0 and 1 do not update either lock variable to the locked state. It is determined that computation units 0 and 1 end the execution operation of the target kernel function kernel2, and schedule computation unit 4 to execute the subsequent processor task, such as executing the operation of the next kernel function kernel3.

[0184] Next, the execution process of the kernel function kernel3 corresponding to the computation operation A2×W0 is described.

[0185] Similar to the kernel function process described above, the contention work blocks include work blocks A2_10 and A2_11, while the other non-contention work blocks include work blocks A2_1-A2_9. Tasks are allocated to the non-contention work blocks according to a static, pre-defined mapping relationship.

[0186] like Figure 5 As shown, for non-contested work blocks A2_1-A2_9, tasks are still allocated according to the aforementioned preset static mapping relationship, for example, as... Figure 5 As shown, computing unit 0 executes non-contested work blocks A2_0 and A2_5, computing unit 1 executes non-contested work blocks A2_1 and A2_6, and so on.

[0187] After the computation unit 4 finishes executing the target work block A2_9, the computation unit 4 uses atomic operations to compete for the two lock variables corresponding to the work blocks A2_10 and A2_11 in a loop, in order to try to update either lock variable to the locked state.

[0188] For example, computation unit 4 iterates through the two lock variables and performs atomic comparison and swap operations on the lock variables, such as atomicCAS, as detailed in the previous embodiment, which will not be repeated here. If the atomic comparison and swap operation performed with the lock variable corresponding to work block A2_10 returns 0, it indicates that the operation was successful. Computation unit 4 has written the identifier value 5 into the lock variable corresponding to work block A2_10, confirming that computation unit 4 has obtained the qualification to execute the computation task of work block A2_10. Computation unit 4 withdraws from the competition and executes the computation task of work block A2_10.

[0189] After computation units 0 through 3 have completed their respective target work blocks, each of the four computation units uses atomic operations in a loop to compete for two lock variables in an attempt to update either lock variable to a locked state.

[0190] For example, after computation unit 0 performs an atomic comparison and swap operation with the lock variable corresponding to work block A2_10, it returns an operation failure. Then, computation unit 0 performs an atomic comparison and swap operation with the lock variable corresponding to work block A2_11. For example, if it returns 0, it means that the operation was successful. Computation unit 0 writes the flag value 1 into the lock variable corresponding to work block A2_11, confirming that computation unit 0 has obtained the qualification to execute the computation task of work block A2_11. Computation unit 0 withdraws from the competition and executes the computation task of work block A2_11.

[0191] At this point, the lock variable corresponding to work block A2_10 has a value of 5, and the lock variable corresponding to work block A2_11 has a value of 1; both are locked. When other computation units 1-3 attempt to perform atomic comparison and swap operations, all operations return failure. It is determined that computation units 1-3 have finished executing the target kernel function kernel3, and are scheduled to execute subsequent processor tasks, such as directly starting the execution of the next kernel function kernel4. Figure 5 In the 8th wave, computational units 1-3 execute the computational tasks of work blocks A3_1-A3_3.

[0192] After completing work block A2_10, computation unit 4 also determines to end the operation of the current kernel function kernel3 and begin the operation of the next kernel function kernel4, such as... Figure 5 As shown, in the 8th wave, computing unit 4 performs the computing task of work block A3_4.

[0193] Next, the execution process of the kernel function kernel4 corresponding to the computation operation A3×W0 is described.

[0194] Similar to the kernel function process described above, the contention work blocks include work blocks A3_10 and A3_11, while the other non-contention work blocks include work blocks A3_1-A3_9. Tasks are allocated to the non-contention work blocks according to a static, pre-defined mapping relationship.

[0195] like Figure 5 As shown, for non-contested work blocks A3_1-A3_9, tasks are still allocated according to the aforementioned preset static mapping relationship, for example, as... Figure 5 As shown, computing unit 0 executes non-contested work blocks A3_0 and A3_5, computing unit 1 executes non-contested work blocks A3_1 and A3_6, and so on.

[0196] After computation unit 1 finishes executing the target work block A3_6, computation unit 1 uses atomic operations to compete for the two lock variables corresponding to work blocks A3_10 and A3_11 in a cyclical manner, in order to attempt to update either lock variable to a locked state.

[0197] For example, computation unit 1 iterates through the two lock variables and performs atomic comparison and swap operations on the lock variables, such as atomicCAS. For details, please refer to the aforementioned embodiment; further explanation is omitted here. If the atomic comparison and swap operation with the lock variable corresponding to work block A3_10 returns 0, it indicates that the operation was successful. Computation unit 1 has written the identifier value 2 into the lock variable corresponding to work block A3_10, confirming that computation unit 1 has obtained the qualification to execute the computation task of work block A3_10. Computation unit 1 then exits the competition and executes the computation task of work block A3_10.

[0198] After computation unit 2 completes the execution of target work block A3_7, it uses atomic operations cyclically to compete for two lock variables, attempting to update either lock variable to a locked state. Referring to a similar process described above, if the atomic comparison and swap operation with the lock variable corresponding to work block A3_11 returns 0, it indicates success, confirming that computation unit 2 has acquired the qualification to execute the computation task of work block A3_11. Computation unit 2 then exits the competition and executes the computation task of work block A3_11.

[0199] After completing their respective target working blocks, computation units 0, 3, and 4 repeatedly use atomic operations to compete for two lock variables. At this point, working blocks A3_10 and A3_11 are both locked. When computation units 0, 3, and 4 attempt to perform atomic comparison and swap operations, all operations return failure. It is determined that computation units 0, 3, and 4 have finished executing the target kernel function kernel4, and are scheduled to execute subsequent processor tasks. Since kernel4 is the last computational operation, the computation units wait for the current computational operation to finish before starting other processor tasks.

[0200] like Figure 3C As shown, in this embodiment, the computational processing can be completed in just 10 waves, compared to... Figure 6 The reduction of two waves from the 12 waves effectively compresses the cumulative long-tail effect of multiple kernel functions in the fusion computing process. This results in a more balanced load across all computing units in the entire fusion task, allowing the total execution time of multiple small kernel functions to approach or even reach the performance of a single large kernel function. This ensures the effectiveness of the fusion computing technology and improves overall hardware utilization and program throughput. Furthermore, in this embodiment, the overhead of the competition mechanism and atomic operations only occurs in the last wave, which has the least computational load. For the majority of normal waves, the execution method is no different from the traditional method, with no additional overhead. This design ensures that while addressing core performance bottlenecks, it does not have a significant negative impact on overall computing performance.

[0201] At least one embodiment of this disclosure also provides a parallel processor. Figure 6 A schematic structural diagram of a parallel processor provided for at least one embodiment of this disclosure.

[0202] like Figure 1 As shown, the parallel processor 100 includes multiple computing units 101.

[0203] The parallel processor can be, for example, a graphics processing unit (GPU) or a general-purpose GPU. When the parallel processor is a GPU or a general-purpose GPU, its specific architecture can be found in [reference needed]. Figure 7 The relevant descriptions will not be repeated here.

[0204] Multiple computing units can execute the same or different computing tasks in parallel.

[0205] The parallel processor 100 is configured to perform computational processing operations, wherein the computational processing operations include multiple computational operations executed sequentially, the input data of the computational processing operations are divided into multiple data blocks, and each computational operation is used to process one of the multiple data blocks.

[0206] When the parallel processor 100 executes each computational operation, it includes performing the following operations: determining at least one competing working block among the multiple working blocks based on the number of multiple working blocks included in the input data of the target kernel function corresponding to the computational operation and the number of the multiple computing units; and having the multiple computing units process the computational tasks of the multiple working blocks in a pipelined parallel manner to complete the computational operation.

[0207] For the at least one competing work block, a competition mechanism is used to dynamically determine the allocation relationship between multiple computing units and the computing tasks of the at least one competing work block. The competition mechanism includes the multiple computing units competing for the qualification to execute the computing tasks of the at least one competing work block, and the computing unit that obtains the qualification to execute the computing tasks of the corresponding competing work block.

[0208] For example, in some embodiments, at least one wave corresponding to the target kernel function includes a competition wave, and the working blocks in the competition wave serve as the at least one competing working block. For any one of the plurality of computing units, after the computing unit completes the computing task of the target working block, atomic operations are used cyclically to compete for the execution qualification of the at least one competing working block, wherein the target working block includes the working blocks in the wave preceding the first competition wave; in response to any computing unit obtaining the execution qualification of any competing working block, the computing unit executes the computing task of the any competing working block; in response to any computing unit not obtaining the execution qualification of any competing working block, the computing unit is scheduled to execute subsequent processor tasks.

[0209] For example, in some embodiments, when a parallel processor dynamically determines the allocation relationship between multiple computing units and the computing tasks of at least one competing work block through a competition mechanism, the following operations are performed: initializing a competition workspace, wherein the competition workspace includes R lock variables initialized to an unlocked state, where R is a positive integer representing the number of the at least one competing work block, and the R lock variables correspond one-to-one with the at least one competing work block; the multiple computing units cyclically use atomic operations to compete for the R lock variables in the competition workspace to determine whether any lock variable can be updated from the unlocked state to the locked state; determining that a first computing unit that updates a lock variable to the locked state obtains the qualification to execute the computing task of the competing work block corresponding to the lock variable, the first computing unit withdraws from the competition, and executes the computing task of the competing work block corresponding to the lock variable; determining that a second computing unit that has not updated any lock variable to the locked state ends the execution operation of the target kernel function, and scheduling the second computing unit to execute subsequent processor tasks.

[0210] For details regarding the specific process by which a parallel processor dynamically determines the allocation relationship between computing units and corresponding computing tasks of work blocks through a competition mechanism, please refer to the description of the relevant embodiments in the aforementioned data processing method, which will not be repeated here.

[0211] For example, computational processing operations and communication processing operations with data dependencies are executed in parallel to realize a computational fusion operator. The communication processing operations include multiple data copy operations, and at least one computational operation and at least one data copy operation are executed in parallel. For details regarding the computational fusion operator, please refer to the descriptions of the relevant embodiments in the foregoing data processing method; they will not be repeated here.

[0212] Figure 7 A schematic structural diagram of a multiprocessor system provided for at least one embodiment of this disclosure.

[0213] like Figure 7 The multiprocessor system 200 shown has multiple computing nodes. Of course, it is understood that a multiprocessor system may include only one computing node, and those skilled in the art can configure it as needed.

[0214] For example, such as Figure 7 As shown, each processor can have its own memory, and each processor manages its own memory independently.

[0215] A parallel processor can be any one of the multiple processors included in a multiprocessor system.

[0216] like Figure 8As shown, each processor can access the memory of other processors through the bus, and multiple processors can be interconnected in pairs through the bus.

[0217] Multiple processors in a compute node are configured to jointly execute a processing task, and each processor is configured to perform a part of that processing task.

[0218] For example, in at least one embodiment of this disclosure, the computation task can be split using tensor parallelism or sequence parallelism, dividing the complete activation tensor and weight tensor into multiple sub-tensors. Each sub-task performs the relevant operations on the corresponding sub-tensor, and the multiple sub-tasks are distributed to multiple processors for parallel execution. Of course, this disclosure is not limited to this, and those skilled in the art can also use other methods to split the computation task and distribute it to multiple processors for parallel execution.

[0219] It should be noted that in at least one embodiment of this disclosure, the multiprocessor system may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited and can be constructed from analog devices, digital chips, or other suitable methods according to circuit principles.

[0220] For example, a multiprocessor system can be implemented in hardware, software, or a combination of both, and this disclosure does not impose any specific limitations on this.

[0221] In some embodiments, the multiprocessor system provided in at least one embodiment of this disclosure can be a single-machine multi-card configuration, where multiple GPU cards are integrated in a single server or workstation, and inter-card communication is achieved through a bus (e.g., PCIe bus, AMD bus, etc.). For example, the multiprocessor system can also be a multi-machine multi-card configuration, where multiple single-machine multi-card servers are interconnected through a high-speed network to form a cluster. Each card can not only communicate with other cards on its own machine but also interact with cards on other nodes via the network. For example, the multiprocessor system can also be a TPU cluster, a multi-FPGA accelerator card system, etc., and this disclosure does not impose specific limitations in these areas.

[0222] For example, this multiprocessor system overcomes the limitations of single-card computing power or memory through parallel computing, and is therefore widely used in scenarios requiring large-scale data processing, high-throughput computing, or low-latency parallel tasks. Applications include deep learning and large model training, high-performance computing, large-scale data processing and AI inference, real-time rendering and visualization, and other fields such as large model training, scientific computing, high-concurrency AI services, and real-time rendering.

[0223] The parallel processor provided in at least one embodiment of this disclosure can effectively compress the cumulative long-tail effect of multiple kernel functions in the fusion of computing and general computing, making the load of all computing units more balanced in the entire fusion task. The total execution time of multiple small kernel functions can approach or even reach the performance of a single large kernel function, ensuring the effectiveness of the fusion of computing and general computing technology and improving the overall hardware utilization and program throughput.

[0224] Figure 8 This is a schematic structural diagram of an electronic device provided for at least one embodiment of the present disclosure.

[0225] like Figure 9 As shown, the electronic device 200 provided in at least one embodiment of this disclosure includes a parallel processor 100. For a description of the parallel processor 100, please refer to the foregoing introduction of the parallel processor, which will not be repeated here.

[0226] For example, depending on actual needs, the electronic device can be a deep learning workstation equipped with a multi-processor system, a data center server, a dedicated AI server, etc. Alternatively, the electronic device can also be a supercomputing center cluster, a cloud service provider's AI cluster, etc. This disclosure does not impose specific limitations in this regard.

[0227] Electronic devices may include other hardware, software, or firmware for working in conjunction with multiprocessor systems, as needed, and this disclosure does not impose specific limitations in this regard.

[0228] For details regarding the technical effects of the electronic device, please refer to the description of the parallel processor in the foregoing embodiments, which will not be repeated here.

[0229] Figure 9 This is a schematic block diagram of another electronic device provided as an embodiment of the present disclosure. Figure 9 As shown, the electronic device 300 is, for example, suitable for implementing the data processing method provided in the embodiments of this disclosure. It should be noted that... Figure 9 The components of the electronic device 300 shown are merely exemplary and not limiting. The electronic device 300 may have other components as needed for the actual application.

[0230] like Figure 9 As shown, the electronic device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in memory to achieve various functions.

[0231] For example, when the computer-readable instruction processing device 301 executes, it can perform one or more steps of the data processing method according to any of the above embodiments. It should be noted that a detailed description of the processing procedure of the data processing method can be found in the relevant descriptions in the embodiments of the above data processing method.

[0232] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc., whereby computer-readable instructions can be loaded from storage device 308 into RAM 303 to execute. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. Various applications and various data, such as various data used and / or generated by applications, may also be stored in the computer-readable storage medium.

[0233] For example, the processing device 301, the read-only memory (ROM) 302, and the random access memory (RAM) 303 are interconnected via a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0234] Typically, the following devices can be connected to the input / output (I / O) interface 305: input devices 306 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 308 including, for example, magnetic tape, hard disk, flash memory, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 10 An electronic device 300 with various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 300 may alternatively implement or possess more or fewer devices. For example, a processing device 301 can control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a device with data processing capabilities and / or program execution capabilities, such as a central processing unit (CPU), a tensor processor (TPU), or a graphics processing unit (GPU). The central processing unit (CPU) may be an x86, ARM, RISC-V architecture, etc. The GPU may be directly integrated into the SOC, directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0235] Figure 10 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of this disclosure. For example, such as ​As shown, storage medium 400 can be a non-transitory computer-readable storage medium on which one or more computer-readable instructions 401 can be stored non-transitory. For example, when the computer-readable instructions 401 are executed by a processor, one or more steps in the data processing method described above can be performed.

[0236] For example, the storage medium 400 can be used in an electronic device 300, such as the storage medium 400 including the storage device 308 in the electronic device 300.

[0237] For example, a storage device may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and a processor may execute these instructions to perform various functions of the processor. Various application programs and various data may also be stored in the storage medium.

[0238] For example, the storage medium may include a memory card for a smartphone, a cache component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0239] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0240] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0241] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0242] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0243] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0244] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0245] The following points should be noted regarding this disclosure:

[0246] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0247] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0248] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A data processing method, characterized in that, Applied to a parallel processor, the parallel processor comprising multiple computing units, The data processing method includes executing computational processing operations through the parallel processor, wherein the computational processing operations include multiple computational operations executed sequentially, the input data of the computational processing operations are divided into multiple data blocks, and each computational operation is used to process one of the multiple data blocks. The plurality of computational operations included in the computational processing operations executed by the parallel processor include: For each computational operation: Based on the number of multiple working blocks included in the input data of the target kernel function corresponding to the computation operation and the number of multiple computation units, at least one competing working block among the multiple working blocks is determined; The computational tasks of the multiple work blocks are processed in a pipelined parallel manner by the multiple computing units to complete the computational operation; Specifically, for the at least one competing work block, a competition mechanism is used to dynamically determine the allocation relationship between the multiple computing units and the computing tasks of the at least one competing work block. The competition mechanism includes the multiple computing units competing for the qualification to execute the computing tasks of the at least one competing work block, and the computing unit that obtains the qualification to execute the computing tasks of the corresponding competing work block.

2. The data processing method according to claim 1, characterized in that, The target kernel function corresponds to at least one wavelet, which includes a competing wavelet, and the working blocks in the competing wavelet are the at least one competing working block. For any one of the plurality of computing units, after the computing unit has completed the computing task of the target working block, atomic operations are used cyclically to compete for the execution qualification of at least one competing working block, wherein the target working block includes the working block in the wave preceding the first competing wave in the at least one wave. In response to any computing unit obtaining the qualification to execute any competing work block, the computing unit executes the computing task of the competing work block. In response to any computing unit failing to qualify for execution of any competing work block, the computing unit is scheduled to execute subsequent processor tasks.

3. The data processing method according to claim 1, characterized in that, The allocation relationship between the computational tasks of the multiple computing units and the at least one competing work block is dynamically determined through a competition mechanism, including: Initialize a contention workspace, wherein the contention workspace includes R lock variables initialized to an unlocked state, where R is a positive integer and represents the number of the at least one contention work block, and the R lock variables correspond one-to-one with the at least one contention work block; The plurality of computing units cyclically use atomic operations to compete for the R lock variables in the competitive workspace to determine whether any lock variable can be updated from the unlocked state to the locked state; The first computing unit that updates a lock variable to the locked state is granted the qualification to execute the computing task of the competing work block corresponding to the lock variable. The first computing unit withdraws from the competition and executes the computing task of the competing work block corresponding to the lock variable. Once it is determined that the second computing unit has not updated any lock variable to the locked state, the execution operation of the target kernel function ends, and the second computing unit is scheduled to execute subsequent processor tasks.

4. The data processing method according to claim 3, characterized in that, The plurality of computing units cyclically use atomic operations to compete for the R lock variables in the competitive workspace to determine whether any lock variable can be updated from the unlocked state to the locked state, including: For any one of the plurality of computing units: The R lock variables are iterated over in a loop, and the atomic operation is performed on each of the R lock variables. The atomic operation includes, in response to determining that the current lock variable is in the unlocked state, swapping the identifier value corresponding to any one of the computing units to the current lock variable and returning a success operation, and in response to determining that the current lock variable is not in the unlocked state, returning a failure operation. In response to the atomic operation performed with the current lock variable returning a success value, it is determined that either computational unit updates the current lock variable to the locked state. In response to the atomic operation performed with the current lock variable returning an operation failure, and determining that any one of the computation units has not updated the current lock variable to the locked state, the computation unit continues to perform the atomic operation with the next lock variable until all R lock variables have been traversed.

5. The data processing method according to claim 4, characterized in that, The atomic operations include atomic comparison and exchange operations. The atom comparison and exchange operation includes: Compare the current value of the lock variable with the preset value corresponding to the unlocked state. In response to the fact that the value of the current lock variable is the same as the preset value, the current lock variable is determined to be in the unlocked state, and the identifier value corresponding to any one of the computing units is swapped to the current lock variable. The preset value is then returned to indicate that the operation was successful. In response to the fact that the value of the current lock variable is different from the preset value, it is determined that the current lock variable is not in the unlocked state, and the value of the current lock variable is returned to indicate that the operation has failed. The identifier value corresponding to each of the plurality of computing units is different from the preset value.

6. The data processing method according to claim 3, characterized in that, Initialize the contention workspace, including: A global memory region is allocated on the memory of the parallel processor to serve as the contention workspace; In the contested workspace, the R lock variables are set, wherein the R lock variables are initialized to the unlocked state.

7. The data processing method according to claim 1, characterized in that, Based on the number of multiple working blocks included in the input data of the target kernel function corresponding to the computational operation and the number of multiple computational units, at least one competing working block among the multiple working blocks is determined, including: Based on the number N of the plurality of working blocks and the number M of the plurality of computing units, the theoretical number W of at least one wave corresponding to the target kernel function is determined, wherein each wave includes the computing tasks of M working blocks performed in parallel by M computing units to complete one round of parallel tasks. Based on the theoretical quantity W, determine the competing waves in the at least one wave; The working block in the competitive wave is identified as the at least one competing working block. Where M, N, and W are all positive integers.

8. The data processing method according to claim 7, characterized in that, Based on the theoretical quantity W, determining the competing waves in the at least one wavelet includes: In response to W > 1, the m-th wave executed out of W waves and all waves following the m-th wave are determined as the competing waves, where m is a preset positive integer less than or equal to W. In response to W=1, the at least one wavelet is determined to be the competing wavelet.

9. The data processing method according to claim 2, characterized in that, Scheduling any one of the computing units to execute subsequent processor tasks includes: In response to the fact that the computation operation corresponding to the target kernel function is not the last computation operation executed among the plurality of computation operations, the computation unit is scheduled to immediately execute the computation task in the kernel function corresponding to the next computation operation.

10. The data processing method according to claim 1, characterized in that, For the other work blocks among the plurality of work blocks, excluding the at least one competing work block, the allocation relationship between the plurality of computing units and the computing tasks of the other work blocks is determined according to a static preset mapping relationship.

11. The data processing method according to any one of claims 1-10, characterized in that, The computational processing operations are executed in parallel with the communication processing operations that have data dependencies to realize the integrated computing operator. The communication processing operation includes multiple data copy operations, and at least one of the multiple computation operations is executed in parallel with at least one data copy operation.

12. The data processing method according to claim 11, characterized in that, The parallel processor is any one of a plurality of processors included in a multiprocessor system, the plurality of processors being configured to jointly execute a processing task, and each processor being configured to execute a portion of the processing task. The processing task includes executing the general computing fusion operator, and the computational processing operation includes a portion of the processing task in the general computing fusion operator that is executed by the parallel processor.

13. A parallel processor, characterized in that, Includes multiple computing units, The parallel processor is configured to perform computational processing operations, wherein the computational processing operations include multiple computational operations executed sequentially, the input data of the computational processing operations is divided into multiple data blocks, and each computational operation is used to process one of the multiple data blocks. When the parallel processor executes each computational operation, it includes performing the following operations: Based on the number of multiple working blocks included in the input data of the target kernel function corresponding to the computation operation and the number of multiple computation units, at least one competing working block among the multiple working blocks is determined; The computational tasks of the multiple work blocks are processed in a pipelined parallel manner by the multiple computing units to complete the computational operation; Specifically, for the at least one competing work block, a competition mechanism is used to dynamically determine the allocation relationship between the multiple computing units and the computing tasks of the at least one competing work block. The competition mechanism includes the multiple computing units competing for the qualification to execute the computing tasks of the at least one competing work block, and the computing unit that obtains the qualification to execute the computing tasks of the corresponding competing work block.

14. An electronic device, characterized in that, Including the parallel processor as described in claim 13.

15. An electronic device, characterized in that, include: Memory stores computer-executable instructions non-transiently; The processor is configured to run computer-executable instructions. The computer-executable instructions are executed by the processor to implement the data processing method according to any one of claims 1-12.

16. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they implement the data processing method according to any one of claims 1-12.

Images