Calculation task processing method and device, calculation equipment and storage medium

Abstract

The invention provides a computing task processing method and device, electronic equipment and a storage medium, and the method comprises the steps: determining a plurality of sub-tasks in a computing task as a task cluster, and distributing the plurality of sub-tasks in the task cluster to a plurality of computing units in the computing cluster; wherein a plurality of sub-tasks in the task cluster multiplex the same task data, and the same task data is divided into a plurality of data blocks, so that each data block is respectively processed by different computing units in different execution rounds; after any calculation unit in the calculation cluster finishes the first execution round, transmitting data blocks processed by any calculation unit in the first execution round to a first adjacent calculation unit; moreover, any computing unit receives the data blocks processed by the second adjacent computing unit in the first execution round so as to process the second execution round. According to the mode, the bottleneck problem of global bandwidth is avoided.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, computing device, and storage medium for processing computing tasks. Background Technology

[0002] Computing systems require the execution of numerous large-scale computational tasks. For example, massively parallel computing tasks such as matrix multiplication and convolution operations are fundamental to many core application areas, including artificial intelligence, scientific computing, and graphics rendering. These types of computational tasks are typically characterized by computational intensity and large data volumes.

[0003] In related technologies, to improve computational efficiency, a complex computational task can be decomposed into multiple smaller subtasks. Correspondingly, each subtask runs independently, thereby improving efficiency through parallelization. Summary of the Invention

[0004] This disclosure provides a method, apparatus, computing device, and storage medium for processing computing tasks.

[0005] In a first aspect, this disclosure provides a method for processing computational tasks, the method being applied to a computing device including a computing cluster, the computing cluster comprising a plurality of interconnected computing units; the method comprising:

[0006] Multiple subtasks in the computing task are defined as a task cluster, and multiple subtasks in the task cluster are assigned to multiple computing units in the computing cluster; wherein, multiple subtasks in the task cluster reuse the same task data, and the same task data is divided into multiple data blocks, so that each data block is processed by different computing units in different execution rounds.

[0007] After the first execution round ends, any computing unit in the computing cluster transmits the data block processed by the computing unit in the first execution round to the first adjacent computing unit; and the computing unit receives the data block processed by the second adjacent computing unit in the first execution round to process the second execution round until the subtask assigned to the computing unit is completed; wherein, the second execution round is the next round after the first execution round.

[0008] In a second aspect, this disclosure provides a processing apparatus for computing tasks, the apparatus being applied to a computing device including a computing cluster, the computing cluster comprising a plurality of interconnected computing units; the apparatus includes:

[0009] The allocation module is adapted to determine multiple subtasks in the computing task as a task cluster, and allocate multiple subtasks in the task cluster to multiple computing units in the computing cluster; wherein, multiple subtasks in the task cluster reuse the same task data, and the same task data is divided into multiple data blocks, so that each data block is processed by different computing units in different execution rounds.

[0010] Wherein, after the first execution round ends, any computing unit in the computing cluster transmits the data block processed by the computing unit in the first execution round to the first adjacent computing unit; and, the computing unit receives the data block processed by the second adjacent computing unit in the first execution round to process the second execution round, until the subtask assigned to the computing unit is completed; wherein, the second execution round is the next round after the first execution round.

[0011] Thirdly, this disclosure provides a computing device, including: a task scheduling unit and a computing cluster connected to the task scheduling unit, wherein the computing cluster includes a plurality of interconnected computing units;

[0012] The task scheduling unit is used to determine multiple subtasks in the computing task as a task cluster, and to allocate multiple subtasks in the task cluster to multiple computing units in the computing cluster; wherein, multiple subtasks in the task cluster reuse the same task data, and the same task data is divided into multiple data blocks, so that each data block is processed by different computing units in different execution rounds.

[0013] Any computing unit in the computing cluster is configured to: after the end of the first execution round, transmit the data block processed by the computing unit in the first execution round to the first adjacent computing unit; and receive the data block processed by the second adjacent computing unit in the first execution round to process the second execution round, until the subtask assigned to the computing unit is completed; wherein the second execution round is the next round after the first execution round.

[0014] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method for processing computational tasks.

[0015] Fifthly, this disclosure provides a computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described processing method for the computational task.

[0016] In the embodiments provided in this disclosure, on the one hand, multiple computing units in a computing device are constructed into a set of interconnected computing clusters; on the other hand, multiple subtasks with data reuse relationships are identified as a task cluster, thereby allocating multiple subtasks in the task cluster to multiple computing units in the computing cluster. This adds an intermediate scheduling layer based on task clusters and computing clusters above the traditional subtask and computing unit calling layer, making the scheduling method between multiple subtasks more flexible. Furthermore, multiple subtasks within a task cluster can reuse the same task data, and this task data is divided into multiple data blocks corresponding to multiple computing units, so that each data block is processed by different computing units in different execution rounds. Therefore, data blocks can be directly transferred between multiple computing units to achieve reuse, significantly reducing the amount of data loaded from global memory by each computing unit, thereby significantly reducing the global memory bandwidth requirement during the execution of the entire task cluster, avoiding the global bandwidth bottleneck problem, and optimizing data processing efficiency.

[0017] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 A flowchart illustrating a method for processing a computational task provided in an embodiment of this disclosure;

[0020] Figure 2 The diagram illustrates the correspondence between computing power, computing cycle, and memory bandwidth.

[0021] Figure 3 A schematic diagram showing performance data when using larger-sized blocks is presented;

[0022] Figure 4 A schematic diagram showing the connection relationships between multiple MPs is provided.

[0023] Figure 5 A schematic diagram of matrix A is shown;

[0024] Figure 6 A schematic diagram of matrix B is shown;

[0025] Figure 7 A schematic diagram of a computing device provided in one embodiment of the present disclosure is shown;

[0026] Figure 8 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] Unless otherwise specified, the various embodiments and features of this disclosure may be combined with each other. As used herein, the term "and / or" includes any and all combinations of one or more of the associated enumerated entries.

[0029] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0030] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0031] Figure 1 This is a flowchart illustrating a method for processing a computing task according to an embodiment of this disclosure. The method is applied to a computing device including a computing cluster, which comprises multiple interconnected computing units. (Refer to...) Figure 1 The method includes:

[0032] Step S110: Determine multiple subtasks in the computation task as a task cluster, and assign multiple subtasks in the task cluster to multiple computation units in the computation cluster; wherein, multiple subtasks in the task cluster reuse the same task data, and the same task data is divided into multiple data blocks, so that each data block is processed by different computation units in different execution rounds.

[0033] In this context, a computing cluster refers to a collection of computing units within a computing device. Multiple computing units within a cluster can be connected via high-speed interconnect structures (such as ring networks or mesh networks) to collaboratively process a common computing task. A computing cluster can correspond to a multiprocessor cluster (MPC) in a GPU or a similar architecture. This application does not limit the specific implementation form of the computing cluster. A computing unit is the basic processing core that constitutes a computing cluster, capable of independently executing instructions and performing data operations. For example, a streaming multiprocessor or tensor core in a GPU, as well as a single DSP core in a multi-DSP system, can all be considered computing units.

[0034] A task cluster refers to a collection of multiple subtasks that are logically grouped together because they need to reuse the same data or have close dependencies. Task clusters can serve as the basic unit of task scheduling, ensuring that multiple subtasks within the same cluster can be scheduled to execute on different computing units within the same cluster.

[0035] In this context, multiple subtasks within a task cluster reuse the same task data. Task data refers to the set of input data that is shared and relied upon by multiple subtasks during the execution of a computation task. Task data can have the following characteristics: the data itself remains unchanged during computation or is only read, but needs to be processed with different other data to produce different results. For example, in matrix multiplication, if a task cluster is responsible for calculating multiple rows of the output matrix, then the corresponding rows in the input matrix can be considered as task data reused by multiple subtasks. Correspondingly, data chunking refers to data units of a size suitable for processing by a single computational unit, obtained by dividing the task data that needs to be reused. For example, data chunks can be subsets obtained by dividing the aforementioned task data according to certain rules (such as uniform division, division by row / column, etc.). The size and shape of each data chunk match the processing capacity of the computational units in the computation cluster, aiming to achieve load balancing.

[0036] The relationship between multiple data blocks and multiple computing units is not a fixed, one-to-one binding, but rather a dynamic many-to-many mapping similar to time-sharing multiplexing and round-robin processing. Specifically, in any given execution round (e.g., the first execution round), multiple data blocks are assigned to multiple computing units for parallel processing. At this time, each computing unit can process one or more specific, non-overlapping data blocks; that is, multiple computing units jointly process multiple data blocks. Furthermore, to ensure that each data block is processed by different computing units in different execution rounds, the data blocks processed by each computing unit are systematically exchanged (round-robined) between adjacent execution rounds through interconnection between computing units.

[0037] An execution round is a unit of time or operation used to describe the progress of a computation task. Typically, a complete computation task is completed through multiple execution rounds. In each round, the computation unit processes the currently held data block and exchanges data at the end of the round. For example, an execution round can be a complete computation cycle in which a computation unit processes its currently held data block. The completion of a subtask requires multiple execution rounds, in which the computation unit processes a specific data block.

[0038] The multiple computing units within a computing cluster can be connected in various ways. For example, they can be connected via a ring interconnect, where multiple computing units are linked end-to-end to form a ring. This method has a simple structure, a single path, and simple control logic, making it suitable for pipelined data transfer and easy to achieve high bandwidth and low latency. Alternatively, they can be connected via a grid interconnect, where multiple computing units are arranged in a two-dimensional grid, with each unit connected to four adjacent units in the east, west, south, and north directions. This method offers greater scalability and is suitable for larger-scale computing arrays. Furthermore, they can be connected via a full interconnect, where any two computing units have a direct path to each other. Those skilled in the art can flexibly select or combine the above interconnect methods according to the size of the computing cluster, the data flow pattern, and the requirements for bandwidth and latency; this application does not limit this selection.

[0039] Step S120: After the first execution round ends, any computing unit in the computing cluster transmits the data block processed by the computing unit in the first execution round to the first adjacent computing unit; and, any computing unit receives the data block processed by the second adjacent computing unit in the first execution round to process the second execution round, until the subtask assigned to any computing unit is completed; wherein, the second execution round is the next round after the first execution round.

[0040] The first execution round can also be understood as the current execution round, specifically a specific computation cycle that the computing unit is currently processing or has just completed. Alternatively, the first execution round can also be understood as the most recently completed or processed execution round. In short, this application does not limit the specific connotation of the first execution round, and the first execution round changes dynamically with the processing process. Typically, each round is associated with a specific data block. Correspondingly, the first adjacent computing unit and the second adjacent computing unit refer to two specific computing units that are directly connected to any computing unit in the interconnection topology of the computing cluster. Here, "first" and "second" are used to distinguish the direction of data transmission. For example, the first adjacent computing unit is usually the target of data transmission (such as the next-hop node of any computing unit), and the second adjacent computing unit is usually the source of data (such as the previous-hop node of any computing unit). Adjacent (directly connected) computing units can directly exchange data through dedicated, high-speed interconnection channels (such as on-chip network links, shared caches, etc.) to avoid inefficient data transfer through global memory.

[0041] For example, in a ring interconnection, multiple computing units within a computing cluster are connected end-to-end via a ring interconnection. In this topology, each computing unit has exactly two neighbors, referred to as the parent computing unit and the child computing unit. Accordingly, the first adjacent computing unit is the child computing unit, and the second adjacent computing unit is the parent computing unit. In practice, all computing units can synchronously transmit their current data blocks in the same direction (e.g., clockwise or counterclockwise). As another example, in a mesh interconnection, multiple computing units within a computing cluster are connected via a mesh or grid network. In this case, a logical ring communication path can be constructed on the physical network through software configuration or hardware routing. Here, the first adjacent computing unit and the second adjacent computing unit are the preceding and following nodes on this logical ring, respectively.

[0042] As those skilled in the art will understand, in the calculation method provided in this application, step S110 can be a pre-executed step, the core of which lies in defining the rules for task partitioning and completing the initial task deployment. The implementation of the above logical function does not depend on a specific physical hardware structure. Therefore, step S110 can be performed by various execution entities with corresponding control or scheduling capabilities, including but not limited to at least one of the following implementation methods:

[0043] (1) Executed by a dedicated control unit integrated within the computing device: In one implementation, the computing device may include a dedicated control unit or task scheduling module. This unit / module serves as the management core of the computing device, responsible for parsing computing tasks, dividing computing tasks into task clusters according to preset strategies (such as data partitioning rules and load balancing algorithms), and statically or dynamically allocating subtasks to various computing units within the computing clusters.

[0044] (2) Execution by a computing unit in the computing cluster: In another implementation, a designated computing unit (such as the main computing unit or the head unit) in the computing cluster can temporarily or permanently act as the control unit. At the start of the computation, the computing unit executes the partitioning and allocation logic and broadcasts the allocation information to other computing units in the cluster. Subsequently, all units enter the parallel computing and data exchange phase.

[0045] (3) Performed by an independent device other than the computing device: In some distributed or heterogeneous computing systems, this step can also be performed by an external device (such as the host CPU, an independent task scheduling server, or a higher-level computing node) that is separate from the computing device. After the external device completes the task division and mapping, it sends the task descriptor and initial data block information to the computing device.

[0046] In the embodiments provided in this disclosure, on the one hand, multiple computing units in a computing device are constructed into a set of interconnected computing clusters; on the other hand, multiple subtasks with data reuse relationships are identified as a task cluster, thereby allocating multiple subtasks in the task cluster to multiple computing units in the computing cluster. This adds an intermediate scheduling layer based on task clusters and computing clusters above the traditional subtask and computing unit calling layer, making the scheduling method between multiple subtasks more flexible. Furthermore, multiple subtasks within a task cluster can reuse the same task data, and this task data is divided into multiple data blocks, so that each data block is processed by different computing units in different execution rounds. Therefore, data blocks can be directly transferred between multiple computing units to achieve reuse, significantly reducing the amount of data loaded from global memory by each computing unit, thereby significantly reducing the global memory bandwidth requirement during the execution of the entire task cluster, avoiding the global bandwidth bottleneck problem, and optimizing data processing efficiency.

[0047] In an alternative implementation, to perform the initial data loading operation, the method further includes the following operation: any computing unit in the computing cluster retrieves the data block it needs to process in the first execution round from global memory. Therefore, by performing ordered, distributed initial data loading before the first execution round, it can be ensured that each computing unit only needs to retrieve the data block it requires in the first round from global memory before computation begins. This evenly distributes the initial memory access pressure across all computing units, avoiding access conflicts, significantly shortening the startup preparation time of the computing cluster, and improving data processing efficiency.

[0048] Global memory refers to a large-capacity storage device within a computing device that can be shared and accessed by all computing clusters and even all computing units, such as the video memory in the processor of a computing task. Global memory is characterized by its large capacity, but its access latency is typically higher compared to the registers or caches within a computing unit. A computing unit can transfer data blocks processed in the first execution round from global memory to its local storage space using memory read / write instructions or Direct Memory Access (DMA) operations. The first execution round can correspond to the first computational operation performed by each computing unit after the start of the entire computing task. The data block corresponding to the first execution round cannot be obtained by exchanging with other computing units, therefore it needs to be initialized and loaded from global memory. In practice, multiple loading relationships between multiple data blocks and multiple computing units can be pre-defined, and loading can be performed based on these relationships.

[0049] In one optional implementation, a subtask may include N execution rounds. Accordingly, if the first execution round is the i-th execution round and the second execution round is the (i+1)-th execution round, the data block processed by the second adjacent computing unit in the i-th execution round is received to process the (i+1)-th execution round; where i and N are natural numbers, and i is less than N; the execution result of the subtask is determined based on the execution results of the N execution rounds. After each execution round, i is incremented by 1, and the above steps are repeated until the N-th execution round is reached. N execution rounds refer to the total number of computation cycles required for a subtask from start to finish. The value of N can be determined by the characteristics of the task cluster and the total number of data blocks. For example, in a computing cluster containing K computing units, if the task data is divided into K data blocks, each subtask typically requires N=K execution rounds to ensure that each data block is processed by the subtask once. The execution result of a round refers to the partial or intermediate result generated by a computing unit in a single execution round (e.g., the i-th round) after performing calculations based on the data blocks held by the computing unit in that round. The execution result of a task refers to the complete calculation result finally output by a subtask after completing all N execution rounds, which is usually obtained by performing preset calculations based on the execution results of each round. Therefore, this method can determine whether the current execution round is the last execution round based on the total number of execution rounds contained in the subtask, thus achieving complete processing of the subtask.

[0050] In one optional implementation, the computation task is used to calculate the preset operation result between the first data array and the second data array, with identical task data determined based on the first data array. Correspondingly, when any computation unit processes the (i+1)th execution round, it takes the data block processed by the second adjacent computation unit in the i-th execution round as the first data block and determines the second data block in the second data array corresponding to the first data block; it retrieves the second data block from global memory, and any computation unit processes the (i+1)th execution round based on the second data block and the first data block. Here, the first data array and the second data array can be two sets of input data participating in the preset operation. For example, in matrix multiplication, the first data array and the second data array are the left matrix and the right matrix, respectively; in convolution, the first data array and the second data array are the input feature map and the convolution kernel, respectively. The preset operation result refers to the result calculated between the first data array and the second data array according to the set operation rules, such as the product matrix of matrix multiplication, the output feature map of convolution, etc. The operational rules can include various types of operations such as matrix multiplication, convolution, product operations in attention mechanisms, and element-wise binary operations, which are not limited in this disclosure. Correspondingly, the first data block is the data block processed by the second adjacent computation unit in the i-th execution round. Therefore, the first data block is dynamically changing and differs as the rounds progress. The second data block refers to the portion of data selected from the second data array required for operations with the first data block in the current round. Typically, the second data block and the first data block have a fixed correspondence. For example, this correspondence can be determined by the characteristics of matrix multiplication or convolution operations.

[0051] For example, the first data array can be a first matrix containing multiple first vectors, and the second data array can be a second matrix containing multiple second vectors. The preset operation result is the result of matrix multiplication. Wherein, if the first vector is a row vector, the second vector is a column vector; and if the first vector is a column vector, the second vector is a row vector. Thus, the first matrix and the second matrix are two input matrices participating in the matrix multiplication. For example, in a fully connected layer of a neural network, the first matrix can be an activation matrix, and the second matrix can be a weight matrix.

[0052] The first data block can be determined based on at least one first vector in the first matrix, and the second data block can be determined based on at least one second vector in the second matrix. For example, the first data block may include at least one first vector in the first matrix; the second data block may include at least one second vector in the second matrix corresponding to the operation performed with at least one first vector. Therefore, given that the first vector is determined, the second vector is the vector that needs to be multiplied with the first vector based on the rules of matrix multiplication.

[0053] In one implementation, the first vector is a row vector. In this case, the same task data consists of at least one row of the first matrix, which is divided into multiple row vector blocks. These row vector blocks are cyclically shifted between computational units. The second vector is a column vector. In each round, each computational unit loads the corresponding column vector block from the second matrix based on the currently received row vector block for computation. Similarly, the first vector can also be a column vector. In this case, the same task data consists of several columns of the first matrix.

[0054] In one optional implementation, the computing device includes multiple computing clusters, and the computing task includes multiple task clusters. Multiple subtasks within a computing task are defined as task clusters. When assigning multiple subtasks from a task cluster to multiple computing units within a computing cluster, the computing task is divided into multiple task clusters based on multiple first vectors and multiple second vectors within the computing task. These multiple task clusters are then assigned to multiple computing clusters, such that the multiple subtasks within each task cluster correspond to the multiple computing units within their respective computing clusters. Specifically, multiple task clusters correspond to multiple first vectors and multiple second vectors, and multiple computing clusters correspond to multiple task clusters.

[0055] In practice, multiple task clusters can be assigned to multiple computation clusters, with each computation cluster being assigned at least one task cluster. This ensures that multiple subtasks within the same task cluster correspond to multiple computation units within the same computation cluster. Each task cluster can be defined as a set of operations that computes all valid combinations between at least one first vector and at least one second vector.

[0056] For example, a task cluster can be defined around a computational task for a specific set of vectors (a submatrix). Specifically, multiple task clusters can be scheduled and assigned to multiple computational clusters, where a computational cluster is assigned one or more task clusters. For each task cluster assigned to any computational cluster, the multiple subtasks contained within that task cluster are further distributed to multiple computational units within that cluster for execution. In this application, the relationship between task clusters and vectors can be that the computational scope of a task cluster is defined by a specific subset of vectors. For example, in row-partitioned matrix multiplication, a task cluster corresponds to a row block of a first matrix (containing multiple consecutive row vectors), and its task is to compute the product of that row block and the entire second matrix.

[0057] The correspondence between computation clusters and task clusters is used to characterize the result of task scheduling, and can be one-to-one, many-to-one, or other various correspondences. For example, task clusters and computation clusters can have a one-to-one correspondence, that is, the number of task clusters and computation clusters are equal, and each task cluster corresponds to one computation cluster; or, task clusters and computation clusters can have a many-to-one relationship, that is, the number of task clusters is greater than the number of computation clusters, thus at least two task clusters are assigned to the same computation cluster.

[0058] The correspondence between subtasks and computing units characterizes the mapping method for task execution. Specifically, the finest-grained computational task within a task cluster (such as the dot product of a single row vector and a single column vector) is statically or dynamically bound to a specific physical computing core within the cluster for execution. Preferably, a one-to-one mapping can be used to achieve load balancing. Of course, when the number of subtasks exceeds the number of computing units, multiple subtasks can be mapped to the same computing unit. In summary, this application does not limit the above-described correspondence method; those skilled in the art can flexibly choose various correspondence methods, such as one-to-one or many-to-one, according to actual needs.

[0059] In this approach, considering that the processing power of a single computing cluster may not be sufficient to complete all computations in a timely manner, multiple task clusters and computing clusters are configured so that all computing clusters can process their respective assigned task clusters in parallel, thereby improving the parallelism at the task level and shortening the overall computation time.

[0060] When allocating task clusters to computing clusters, several allocation strategies can be employed. One approach is through static one-to-one mapping. For example, during task initialization, the task scheduling unit in the computing device statically and one-to-one allocates the generated task clusters to each computing cluster based on their availability. Another approach is through dynamic scheduling. For instance, task clusters can be added to a shared task pool, and idle computing clusters can actively pull task clusters from the pool and execute them.

[0061] In one optional implementation, each computing unit has at least two levels of cache space. When transferring a data block processed by any computing unit in the current execution round to a first adjacent computing unit, the data block processed in the current execution round is retrieved from the first-level cache space of the computing unit and then transferred to the first adjacent computing unit. Correspondingly, after receiving a data block processed by a second adjacent computing unit in the current execution round, the data block processed by the second adjacent computing unit in the current execution round is further stored in the second-level cache space of any computing unit. This hierarchical storage strategy reduces invalid movement between caches, lowers access latency, and provides hardware support for high-frequency, low-latency data cyclic shifting, thereby improving overall computing efficiency. Therefore, in this application, multi-level caches can be configured according to different execution logics such as data reception, data transmission, and data processing, thereby improving data processing efficiency.

[0062] Preferably, to improve data transmission efficiency, multiple computing units in the computing cluster are connected via a ring interconnection, and each computing unit has a superior computing unit and a subordinate computing unit. Accordingly, the first adjacent computing unit is the subordinate computing unit of any given computing unit, and the second adjacent computing unit is the superior computing unit of any given computing unit. The ring interconnection refers to a topology that connects multiple computing units into a ring network via communication links. Each computing unit is directly connected to only two adjacent units, and data is transmitted unidirectionally or bidirectionally along the ring. The superior computing unit is typically the data source for the current computing unit, and the subordinate computing unit is typically the data destination for the current computing unit. The computing device may include a processor for the computing task. The first matrix may include an activation matrix for the fully connected layer, and the second matrix may include a weight matrix.

[0063] To facilitate understanding, an example is provided below to illustrate the specific implementation details of the computational task processing method provided in this application. This example involves devices such as GPUs, GPGPUs, and AI accelerators, and covers technologies related to data storage, data communication, distributed systems, and programming models.

[0064] With the development of AI and the rapid application of large-scale models in various fields, the demand for hardware computing power and bandwidth is increasing. To enhance computing power, more computing cores can be added to run more tasks simultaneously. However, in scenarios where the number of tasks is limited, or where there is a need to improve the performance of a single task, the above solutions cannot meet the requirements.

[0065] To address these issues, another approach aims to enhance the performance of individual computational cores. For most AI models, the most commonly used model layers are those used to implement matrix multiplication operations. For example, in a multilayer perceptron, in each layer, the activation values ​​(such as activation vectors or activation matrices) input from the previous layer are multiplied by the weight matrix to obtain the activation values ​​output by the current layer, which are then used as the input activation values ​​for the next layer.

[0066] Typically, to improve parallelism, the operation data for each layer is divided into matrix blocks of the same size. These matrix blocks are then distributed across the entire GPU or GPGPU. The size of the matrix block is related to the resources of the MP (Multiprocessor, i.e., the computing core of the GPU). Here, MP is a specific form of computing unit.

[0067] In a GPU, there are typically multiple MPCs (Multi-Processor Clusters), each MPC further includes multiple MPXs (Multi-Processor Sub-clusters), and each MPX further includes at least two MPs (Multiprocessors). Each MP may further include a TCE (Tensor Compute Engine) and an LMS (Local Memory Store).

[0068] For example, each MPC consists of four MPXs, and each MPX consists of two MPs. The register size of an MP is 512KB, and the size of an LMS is 256KB. An MP is a basic computing unit, and threads used to perform computational tasks are organized into thread groups, which are then distributed to each MP.

[0069] Taking a matrix multiplication operation of size 128x128x64 (tile_M×tile_N×tile_K) as an example, assuming the computing power (i.e., the number of computing units) of TCE in each MP is 2048, 4096, and 8192 respectively, the memory bandwidth requirements are as follows: Figure 2 As shown. Figure 2 This illustrates the correspondence between computing power, computation cycle, and memory bandwidth. The matrix calculation can be in the form of... From... Figure 2 It can be seen that when the computing power is increased to 8192, the memory bandwidth can no longer be increased, thus failing to meet the computing power requirements.

[0070] This shows that as computing power increases, computation speed increases significantly (cycle time decreases). However, the demand for memory bandwidth increases dramatically. However, memory bandwidth is physically limited (chip pin count, power consumption, cost, etc.), therefore, the rate at which memory bandwidth can be increased is far from meeting the demand. In conclusion, the methods for improving computing power are constrained by memory bandwidth.

[0071] To reduce the bandwidth requirements of global memory, one solution is to use larger tiles, such as 256x256x64. Figure 3 A schematic diagram illustrating performance data when using larger-sized blocks is shown. For example, Figure 3 This illustrates the latency hiding of a single MP under different computing power levels. For example... Figure 3 As shown, the data format is FP16, and the matrix data requires 64KB of LMS space. For a 256KB LMS size, it can accommodate a maximum of three levels of cache. From... Figure 3 It can be seen that when the computing power reaches 8192, the bandwidth requirement between MP and LLC is 128B, which cannot be further met by increasing the bandwidth and the size of LMS to satisfy the growing computing power demand. Figure 3 The L2 bandwidth is the same as the LLC bandwidth. In a L3 cache, two matrix blocks can be read consecutively first, then the first matrix block is calculated while the third matrix block is read. After the first matrix block is calculated, the matrix blocks in the L1 cache can be released, and the remaining matrix blocks can be read at the same time.

[0072] Therefore, among the aforementioned technologies, increasing memory access bandwidth significantly increases the complexity of the NOC (Normally Integrating Cache), resulting in larger area and higher power consumption. Another approach is to add more levels of cache; however, this also leads to larger area and higher power consumption, and fine-grained control over cached data reuse becomes difficult.

[0073] In summary, regardless of the solution employed, performance bottlenecks arise due to limited memory bandwidth in all related technologies. Therefore, avoiding insufficient memory bandwidth has become a pressing technical challenge.

[0074] This example implements a distributed storage data sharing scheme and a corresponding programming model, which meets the bandwidth requirements of computing power growth by reusing data between MPs without increasing memory access bandwidth.

[0075] In the operation of multiplying matrix A by matrix B, for matrix blocks in the same row, the same matrix A data needs to be read; for matrix blocks in the same column, the same matrix B data needs to be read. Different matrix blocks are distributed across different storage media (MPs). If MPs can share data, the bandwidth requirements for storage can be reduced. Therefore, this example implements a distributed LMS design. Figure 4 A schematic diagram showing the connection relationships between multiple MPs is provided. Figure 4 As shown, each MP has an independent LMS, and different MPs within the MPC are interconnected through rings. Commands enable mutual access between the LMSs of different MPs, thereby achieving data sharing. Compared to xbar interconnection, the ring structure is simpler, offers significantly higher bandwidth, and has a smaller footprint.

[0076] In this example, the matrix data is a specific implementation of the task data mentioned above, and the multiple matrix blocks in the matrix operations are a specific representation of the multiple subtasks mentioned above. Besides matrix operations, subtasks can also be further subdivided tasks within other computational tasks such as convolution operations and tensor operations. Figure 4 The diagram shows a 4-MPring structure. This example does not limit the number of MPs connected to the ring.

[0077] In typical matrix multiplication operations, a commonly used tile size is 128x128x64. Taking a TCE computing power of 4096 as an example, it takes 64 cycles to complete the calculation of one tile, requiring 128 bytes of storage bandwidth. However, MP0 to MP3 reuse matrix A, and each MP reads its own matrix B. Therefore, the ring bandwidth only needs to be 64 bytes, and the storage bandwidth can be reduced from 128 bytes to below 64 bytes.

[0078] The GPU distributes different tiles to different MPs in the form of workgroups. To ensure that adjacent tiles can be distributed to MPs connected by rings in the same MPC as expected, a new level can be added during the scheduling process: workgroup_cluster. Each ring-connected MP has a unique first identifier (i.e., mp_id_of_ring). Through this first identifier and the identifier of the corresponding workgroup_cluster_id, the corresponding LMS on the corresponding MP can be determined.

[0079] To access different MPs on the ring, the following data commands can be used:

[0080] RING.ST lms_addr, async_bar, size, workgroup_id_in_cluster.

[0081] Here, lms_addr is used to specify the LMS address of the target MP, async_bar is used to indicate whether the target MP tracking data has arrived, size is used to specify the data volume, and workgroup_id_id_cluster is used to convert to the target MP ID.

[0082] In one specific example, the hardware device includes an MP cluster consisting of four MPs: MP0, MP1, MP2, and MP3. The four MPs in this MP cluster are connected via a Ring.

[0083] Suppose the MP cluster needs to read matrices A and B, and obtain matrix C by performing matrix multiplication on matrices A and B. Accordingly, the task allocation for the four MPs is as follows: MP0 is responsible for calculating the N0th column of matrix C, MP1 is responsible for calculating the N1th column of matrix C, MP2 is responsible for calculating the N2nd column of matrix C, and MP3 is responsible for calculating the N3rd column of matrix C.

[0084] Figure 5 A schematic diagram of matrix A is shown. Figure 6 A schematic diagram of matrix B is shown. The matrix calculation process specifically includes the following stages:

[0085] Phase 1: Initial loading and first calculation.

[0086] In this stage, each MP loads its initial data required for the first calculation from global memory. Specifically, MP0 loads (M0, K0) from matrix A and (K0, N0) from matrix B from global memory, and calculates the first component of C00 by A(M0, K0) × B(K0, N0). Therefore, MP0 is responsible for the N0th column of C, so it loads the N0th block of B and initially loads the K0th block of A.

[0087] Similarly, MP1 loads (M0, K1) from matrix A and (K1, N1) from matrix B from global memory, and calculates the first component of C01 using A(M0, K1) × B(K1, N1). Therefore, MP1 is responsible for the N1th column of C, so it loads the N1th block of B and initially loads the K1th block of A. MP2 loads (M0, K2) from matrix A and (K2, N2) from matrix B from global memory, and calculates the first component of C02 using A(M0, K2) × B(K2, N2). Therefore, MP2 is responsible for the N2th column of C, so it loads the N2th block of B and initially loads the K2th block of A. MP3 loads (M0, K3) from matrix A and (N3, K3) from matrix B from global memory, and calculates the first component of C03 using A(M0, K3) × B(K3, N3). Therefore, MP3 is responsible for the N3rd column of C, so the N3rd block of B is loaded, and the K3rd block of A is initially loaded.

[0088] Therefore, after this stage ends, each MP has completed a multiplication and addition operation, and each MP stores a different A matrix block: MP0 stores (M0, K0) in matrix A, MP1 stores (M0, K1) in matrix A, MP2 stores (M0, K2) in matrix A, and MP3 stores (M0, K3) in matrix A.

[0089] Phase 2: First data rotation and calculation.

[0090] MP0 receives (M0,K3) from matrix A from MP3 via Ring, and loads (K3,N0) from matrix B in global memory. It then calculates the second component of C00 by multiplying A(M0,K3) by B(K3,N0). Thus, MP0 loads (K3,N0) from global memory, which corresponds to (M0,K3) in matrix A, to complete a new round of calculation.

[0091] Similarly, MP1 receives (M0, K0) from matrix A of MP0 via Ring, loads (K0, N1) from matrix B in global memory, and calculates the second component of C01 by A(M0, K0) × B(K0, N1). MP2 receives (M0, K1) from matrix A of MP1 via Ring, loads (K1, N2) from matrix B in global memory, and calculates the second component of C02 by A(M0, K1) × B(K1, N2). MP3 receives (M0, K2) from matrix A of MP2 via Ring, loads (K2, N3) from matrix B in global memory, and calculates the second component of C03 by A(M0, K2) × B(K2, N3).

[0092] Therefore, in this stage, each MP transmits the first data block of matrix A, which was stored in the previous stage, to its neighboring MP via a Ring. Each MP then loads the second data block corresponding to the first data block from matrix B in global memory, based on the first data block received from its neighboring MP, thus completing a new round of calculation. After this stage, each MP has completed two accumulation operations, and block A has been rotated once on the Ring.

[0093] Phase 3: Second data rotation and calculation.

[0094] MP0 receives (M0, K2) from matrix A from MP3 via Ring, and loads (K2, N0) from matrix B from global memory. It then calculates the third component of C00 by multiplying A(M0, K2) by B(K2, N0). Thus, MP0 loads (K2, N0) from global memory, which corresponds to (M0, K2) in matrix A, to complete a new round of calculation.

[0095] Similarly, MP1 receives (M0, K3) from matrix A of MP0 via Ring, loads (K3, N1) from matrix B in global memory, and calculates the third component of C01 by A(M0, K3) × B(K3, N1). MP2 receives (M0, K0) from matrix A of MP1 via Ring, loads (K0, N2) from matrix B in global memory, and calculates the third component of C02 by A(M0, K0) × B(K0, N2). MP3 receives (M0, K1) from matrix A of MP2 via Ring, loads (K1, N3) from matrix B in global memory, and calculates the third component of C03 by A(M0, K1) × B(K1, N3).

[0096] Phase 4: Third data rotation and calculation.

[0097] MP0 receives (M0,K1) from matrix A of MP3 via Ring, and loads (K1,N0) from matrix B in global memory. It then calculates the fourth component in C00 using A(M0,K1) ×(K1,N0).

[0098] Similarly, MP1 receives (M0, K2) from matrix A of MP0 via Ring, loads (K2, N1) from matrix B in global memory, and calculates the fourth component of C01 by A(M0, K2) × B(K2, N1). MP2 receives (M0, K3) from matrix A of MP1 via Ring, loads (K3, N2) from matrix B in global memory, and calculates the fourth component of C02 by A(M0, K3) × B(K3, N2). MP3 receives (M0, K0) from matrix A of MP2 via Ring, loads (K0, N3) from matrix B in global memory, and calculates the fourth component of C03 by A(M0, K0) × B(K0, N3).

[0099] Therefore, when K=4, after the above 4 stages, each MP traverses all K blocks (K0, K1, K2, K3) in row M0 of matrix A, and calculates and accumulates them with the corresponding blocks in matrix B.

[0100] The final calculation result on MP0 is expressed by the following formula:

[0101] C00=A(M0,K0)×B(K0,N0)+A(M0,K3)×B(K3,N0)+A(M0,K2)×B(K2,N0)+A(M0,K1)×B(K1,N0).

[0102] Similarly, the final calculation result on MP1 is expressed by the following formula:

[0103] C01=A(M0,K1) × B(K1,N1) + A(M0,K0) × B(K0,N1) + A(M0,K3) × B(K3,N1) + A(M0,K2) × B(K2,N1).

[0104] The final calculation result on MP2 is expressed by the following formula:

[0105] C02 = A(M0,K2) × B(K2,N2) + A(M0,K1) × B(K1,N2) + A(M0,K0) × B(K0,N2) + A(M0,K3) × B(K3,N2).

[0106] The final calculation result on the MP3 is expressed by the following formula:

[0107] C03 = A(M0,K3) × B(K3,N3) + A(M0,K2) × B(K2,N3) + A(M0,K1) × B(K1,N3) + A(M0,K0) × B(K0,N3).

[0108] By combining the above calculation results, we can obtain the M0th row [C00,C01,C02,C03] of the complete output matrix C.

[0109] In summary, the matrix blocks in matrix A flow between multiple MPs in the following order:

[0110] For MP0, in the initial stage, the K0 block of matrix A is obtained from global memory; then, in the second stage, the K3 block of matrix A is obtained from MP3; in the third stage, the K2 block of matrix A is obtained from MP3; and in the fourth stage, the K1 block of matrix A is obtained from MP3.

[0111] For MP1, in the initial stage, the K1 block of matrix A is obtained from global memory; then, in the second stage, the K0 block of matrix A is obtained from MP0; in the third stage, the K3 block of matrix A is obtained from MP0; and in the fourth stage, the K2 block of matrix A is obtained from MP0.

[0112] For MP2, in the initial stage, the K2 block of matrix A is obtained from global memory; then, in the second stage, the K1 block of matrix A is obtained from MP1; in the third stage, the K0 block of matrix A is obtained from MP1; and in the fourth stage, the K3 block of matrix A is obtained from MP1.

[0113] For MP3, in the initial stage, the K3 block of matrix A is obtained from global memory; then, in the second stage, the K2 block of matrix A is obtained from MP2; in the third stage, the K1 block of matrix A is obtained from MP2; and in the fourth stage, the K0 block of matrix A is obtained from MP2.

[0114] As can be seen, the above example optimizes the access process for matrix A. In the traditional approach, four memory accesses (4 MPs × 4 K blocks) are required to read matrix A in order to compute row M0 of matrix C. In this example, each of the four K blocks of matrix A is read from global memory only once (in stage 1) and then shared via the high-speed Ring network. The total number of accesses is reduced to only 4, and the bandwidth requirement is reduced to 1 / 4 of the original.

[0115] In the example above, the CDM (Cluster Distribution Manager) can allocate workgroups belonging to the same cluster to ring-connected MPs according to the new hierarchy `workgroup_cluster`. Therefore, this example implements a ring-based distributed storage solution, which can be used to implement new programming models. This example implements a convolution engine based on existing matrix computation structures, reusing computational logic. Furthermore, direct convolution saves memory access bandwidth compared to the traditional `im2col` implementation, improving computational performance while reducing the complexity of software implementation for users.

[0116] This disclosure also provides a processing apparatus for computing tasks, the apparatus being applied to a computing device including a computing cluster, the computing cluster including a plurality of interconnected computing units; the apparatus includes:

[0117] The allocation module is used to determine multiple subtasks in the computing task as a task cluster, and allocate multiple subtasks in the task cluster to multiple computing units in the computing cluster; wherein, multiple subtasks in the task cluster reuse the same task data, and the same task data is divided into multiple data blocks, so that each data block is processed by different computing units in different execution rounds.

[0118] Wherein, after the first execution round ends, any computing unit in the computing cluster transmits the data block processed by the computing unit in the first execution round to the first adjacent computing unit; and, the computing unit receives the data block processed by the second adjacent computing unit in the first execution round to process the second execution round, until the subtask assigned to the computing unit is completed; wherein, the second execution round is the next round after the first execution round.

[0119] Figure 7 A schematic diagram of a computing device provided in one embodiment of this disclosure is shown, such as... Figure 7 As shown, the computing device includes: a task scheduling unit 71 and a computing cluster 72 connected to the task scheduling unit 71, wherein the computing cluster 72 includes a plurality of interconnected computing units 721;

[0120] The task scheduling unit is used to determine multiple subtasks in the computing task as a task cluster, and to allocate multiple subtasks in the task cluster to multiple computing units in the computing cluster; wherein, multiple subtasks in the task cluster reuse the same task data, and the same task data is divided into multiple data blocks, so that each data block is processed by different computing units in different execution rounds.

[0121] Any computing unit in the computing cluster is configured to: after the end of the first execution round, transmit the data block processed by the computing unit in the first execution round to the first adjacent computing unit; and receive the data block processed by the second adjacent computing unit in the first execution round to process the second execution round, until the subtask assigned to the computing unit is completed; wherein the second execution round is the next round after the first execution round.

[0122] Optionally, the number of computing clusters is multiple, and the multiple computing units in each computing cluster are connected sequentially in a ring interconnection manner, and each computing unit has a higher-level computing unit and a lower-level computing unit;

[0123] Each computing unit communicates with the previous computing unit via a first connection, and with the next computing unit via a second connection. The first connection is used to transmit data from the previous computing unit to the current computing unit, and the second connection is used to transmit data from the current computing unit to the next computing unit.

[0124] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0125] In addition, this disclosure also provides a processing apparatus for computing tasks, an electronic device, and a computer-readable storage medium, all of which can be used to implement any of the computing task processing methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding records in the method section and will not be repeated here.

[0126] Figure 8 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0127] Reference Figure 8 This disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702; and one or more I / O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by the at least one processor 701, and the one or more computer programs are executed by the at least one processor 701 to enable the at least one processor 701 to perform the above-described processing method for computing tasks.

[0128] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the aforementioned method for processing computational tasks. The computer-readable storage medium can be volatile or non-volatile. In this application, the aforementioned method for processing computational tasks can be executed by a hardware system with a specific parallel computing architecture. For example, a processor can refer to a hardware processing device capable of reading and executing the aforementioned computer program code. The processor may include or integrate the aforementioned computing cluster, which consists of multiple interconnected computing units. When the processor executes the computer program on the storage medium, the program code configures or controls the processor, thereby driving the internal computing cluster and each computing unit to collaboratively complete all the aforementioned method steps, such as determining the task cluster, allocating subtasks, and transmitting and processing data blocks. That is, the computing cluster and its computing units can be execution resources or functional components used by the processor to implement the aforementioned method. This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the processing method for the above-described computational task.

[0129] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0130] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0131] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0132] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0133] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0134] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0135] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0136] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0138] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for processing computational tasks, characterized in that, The method is applied to a computing device comprising a computing cluster, the computing cluster comprising a plurality of interconnected computing units; the method includes: Multiple subtasks in the computing task are defined as a task cluster, and multiple subtasks in the task cluster are assigned to multiple computing units in the computing cluster; wherein, multiple subtasks in the task cluster reuse the same task data, and the same task data is divided into multiple data blocks, so that each data block is processed by different computing units in different execution rounds. After the first execution round ends, any computing unit in the computing cluster transmits the data block processed by the computing unit in the first execution round to the first adjacent computing unit; and the computing unit receives the data block processed by the second adjacent computing unit in the first execution round to process the second execution round until the subtask assigned to the computing unit is completed; wherein, the second execution round is the next round after the first execution round.

2. The method according to claim 1, characterized in that, The method further includes: Each computing unit in the computing cluster retrieves the data block processed by that computing unit in the first execution round from global memory.

3. The method according to claim 1, characterized in that, The subtask consists of N execution rounds; The fact that any computing unit receives data blocks processed by the second adjacent computing unit in the first execution round to process the second execution round until the subtask assigned to any computing unit is completed includes: When the first execution round is the i-th execution round and the second execution round is the (i+1)-th execution round, the data block processed by the second adjacent computing unit in the i-th execution round is received to process the (i+1)-th execution round; where i and N are natural numbers, and i is less than N; The execution result of the subtask is determined based on the execution results of the N execution rounds respectively.

4. The method according to claim 3, characterized in that, The computing task is used to calculate a preset operation result between the first data array and the second data array, and the same task data is determined according to the first data array; The step of receiving the data blocks processed by the second adjacent computing unit in the i-th execution round, to process the (i+1)-th execution round, includes: The data block processed by the second adjacent computing unit in the i-th execution round is taken as the first data block, and the second data block in the second data array corresponding to the first data block is determined. The second data block is obtained from the global memory, and each computing unit processes the (i+1)th execution round based on the second data block and the first data block.

5. The method according to claim 4, characterized in that, The first data array is a first matrix containing multiple first vectors, the second data array is a second matrix containing multiple second vectors, and the preset operation result is the result of matrix multiplication. Wherein, if the first vector is a row vector, the second vector is a column vector; if the first vector is a column vector, the second vector is a row vector. The first data block is determined based on at least one first vector in the first matrix, and the second data block is determined based on at least one second vector in the second matrix.

6. The method according to claim 5, characterized in that, The computing device includes multiple computing clusters, and the computing task includes multiple task clusters; the step of determining multiple subtasks in the computing task as task clusters and assigning multiple subtasks in the task clusters to multiple computing units in the computing clusters includes: Based on the multiple first vectors and the multiple second vectors in the computing task, the computing task is divided into multiple task clusters, and the multiple task clusters are respectively assigned to the multiple computing clusters. Each computing cluster is assigned at least one task cluster, so that multiple subtasks in at least one task cluster assigned to the same computing cluster correspond to multiple computing units in the same computing cluster. The plurality of task clusters correspond to the plurality of first vectors and the plurality of second vectors, and the plurality of computation clusters correspond to the plurality of task clusters.

7. The method according to any one of claims 1-6, characterized in that, Each computing unit has at least two levels of cache space, and the step of transmitting the data block processed by any computing unit in the current execution round to the first adjacent computing unit includes: obtaining the data block processed in the current execution round from the first level cache space of any computing unit, and transmitting the data block processed in the current execution round to the first adjacent computing unit; After receiving the data block processed by the second adjacent computing unit in the current execution round, the method further includes: storing the data block processed by the second adjacent computing unit in the current execution round into the second-level cache space of any computing unit.

8. The method according to any one of claims 1-6, characterized in that, The computing cluster contains multiple computing units connected in a ring interconnection manner, and each computing unit has a superior computing unit and a subordinate computing unit. The first adjacent computing unit is the next level computing unit of any computing unit, and the second adjacent computing unit is the previous level computing unit of any computing unit.

9. The method according to any one of claims 5-6, characterized in that, The computing device includes a processor for computing tasks; The first matrix includes the activation matrix of the fully connected layer, and the second matrix includes the weight matrix.

10. A processing apparatus for a computing task, characterized in that, The apparatus is applied to a computing device comprising a computing cluster, the computing cluster comprising a plurality of interconnected computing units; the apparatus includes: The allocation module is used to determine multiple subtasks in the computing task as a task cluster, and allocate multiple subtasks in the task cluster to multiple computing units in the computing cluster; wherein, multiple subtasks in the task cluster reuse the same task data, and the same task data is divided into multiple data blocks, so that each data block is processed by different computing units in different execution rounds. Wherein, after the first execution round ends, any computing unit in the computing cluster transmits the data block processed by the computing unit in the first execution round to the first adjacent computing unit; and, the computing unit receives the data block processed by the second adjacent computing unit in the first execution round to process the second execution round, until the subtask assigned to the computing unit is completed; wherein, the second execution round is the next round after the first execution round.

11. A computing device, characterized in that, include: A task scheduling unit and a computing cluster connected to the task scheduling unit, wherein the computing cluster includes multiple interconnected computing units; The task scheduling unit is used to determine multiple subtasks in the computing task as a task cluster, and to allocate multiple subtasks in the task cluster to multiple computing units in the computing cluster; wherein, multiple subtasks in the task cluster reuse the same task data, and the same task data is divided into multiple data blocks, so that each data block is processed by different computing units in different execution rounds. Any computing unit in the computing cluster is configured to: after the end of the first execution round, transmit the data block processed by the computing unit in the first execution round to the first adjacent computing unit; and receive the data block processed by the second adjacent computing unit in the first execution round to process the second execution round, until the subtask assigned to the computing unit is completed; wherein the second execution round is the next round after the first execution round.

12. The computing device according to claim 11, characterized in that, The number of computing clusters is multiple, and the multiple computing units in each computing cluster are connected sequentially in a ring interconnection manner. Each computing unit has a superior computing unit and a subordinate computing unit. Each computing unit communicates with the previous computing unit via a first connection, and with the next computing unit via a second connection. The first connection is used to transmit data from the previous computing unit to the current computing unit, and the second connection is used to transmit data from the current computing unit to the next computing unit.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1-9.

14. A computer program product comprising computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, characterized in that, When the computer-readable code is run in an electronic device, the processor in the electronic device performs the method of any one of claims 1-9.

Images