Method, apparatus and medium for computing task allocation for multi-core processor

Abstract

A method, apparatus, device, and medium for allocating computational tasks on a multi-core processor are provided. The method, relating to the field of computer science, includes: acquiring a computation graph and resource configuration information of the multi-core processor; clearing historical task allocation information of multiple task units in the computation graph when initiating computational task allocation; determining initial task allocation information for each task unit based on the computation graph and resource configuration information, wherein the task allocation information indicates the partitioning method and processing resources of the computational tasks; propagating the task allocation information among multiple task units according to the data dependencies of the computation graph, making the task allocation information of task units with data dependencies compatible with each other, to obtain target task allocation information; and allocating the computational tasks of each task unit to multiple processing cores based on the target task allocation information. This improves the compatibility between the task allocation information of each task unit.

Description

Technical Field

[0001] Examples of this disclosure relate to the field of computers, and specifically to methods, apparatus, devices, and media for allocating computing tasks to multi-core processors. Background Technology

[0002] As the scale and computational complexity of artificial intelligence models continue to increase, accelerators and other processors typically employ multi-core processing architectures to improve the inference efficiency of neural network models. These architectures utilize multiple processing cores to execute computational tasks such as convolution, matrix multiplication, and element-wise operations in parallel. A typical accelerator may include external memory, on-chip cache, and multiple processing cores, each typically equipped with corresponding computing units and on-chip storage resources. Due to hardware architecture limitations, data access capabilities between different processing cores may be constrained. Therefore, when utilizing multiple processing cores to execute neural network computation tasks, it is necessary to rationally determine the partitioning and allocation methods for each computational task across different processing cores. Summary of the Invention

[0003] In a first aspect, this paper provides a method for allocating computational tasks for a multi-core processor. The method includes: acquiring a computation graph and resource configuration information of the multi-core processor, the computation graph including multiple task units, each task unit including one or more operators and corresponding to a computational task to be executed by the multi-core processor, the resource configuration information indicating the processing resources available for executing the computational task in the multi-core processor and the computational constraints of the processing resources; in response to initiating computational task allocation on the computation graph, clearing historical task allocation information of the multiple task units; determining initial task allocation information for each of the multiple task units based on the computation graph and the resource configuration information, the task allocation information indicating the partitioning method of the computational task and the processing resources used to execute the partitioned computational task; propagating the task allocation information among the multiple task units according to the data dependencies of the computation graph, updating the task allocation information of task units with data dependencies on the determined task units based on the task allocation information of the determined task units to ensure compatibility of the task allocation information of task units with data dependencies, and obtaining target task allocation information for each of the multiple task units; and allocating the computational tasks of each of the multiple task units to multiple processing cores of the multi-core processor based on the target task allocation information.

[0004] In a second aspect of this paper, a computational task allocation apparatus for a multi-core processor is provided. The apparatus includes an acquisition module, a clearing module, a determination module, a propagation module, and an allocation module. The acquisition module is configured to acquire a computation graph and resource configuration information of the multi-core processor. The clearing module is configured to clear historical task allocation information of multiple task units in response to initiating computational task allocation on the computation graph. The determination module is configured to determine initial task allocation information for each of the multiple task units based on the computation graph and the resource configuration information. The propagation module is configured to propagate the task allocation information among the multiple task units according to the data dependencies of the computation graph to obtain target task allocation information for each of the multiple task units. The allocation module is configured to allocate the computational tasks of each of the multiple task units to multiple processing cores of the multi-core processor based on the target task allocation information.

[0005] In a third aspect of this document, an electronic device is provided. The electronic device includes at least one processor and at least one memory. The at least one memory is coupled to the at least one processor and stores instructions for execution by the at least one processor. When executed by the at least one processor, the instructions cause the electronic device to perform the method of the first aspect.

[0006] In a fourth aspect of this document, a computer-readable storage medium is provided. Computer-executable instructions are stored on the computer-readable storage medium. These computer-executable instructions can be executed by a processor to implement the method of the first aspect.

[0007] In this way, compatible task allocation information can be obtained for the current computation graph and the computational constraints of multi-core processors, and computational tasks in the computation graph can be allocated to multiple processing cores based on this task allocation information.

[0008] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the examples in this disclosure, nor is it intended to restrict the scope of the solutions. Other features will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the various examples in this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A schematic diagram of the example environment is shown; Figure 2 A flowchart illustrating an example process for allocating computing tasks for a multi-core processor according to some embodiments of the present disclosure is shown; Figure 3A A flowchart is shown illustrating an example process where multiple input branches are not yet coordinated according to some embodiments of this disclosure; Figure 3B A flowchart illustrating an example process of achieving coordination by reducing the number of cores according to some embodiments of the present disclosure is shown; Figure 3C A flowchart illustrating an example process of reconciliation by inserting an identity convolution according to some embodiments of the present disclosure is shown; Figure 4 A schematic structural block diagram of an example apparatus for computing task allocation for a multi-core processor according to some embodiments of the present disclosure is shown; Figure 5 A block diagram of an electronic device capable of implementing embodiments of the present disclosure is shown. Detailed Implementation

[0010] The examples in this document will now be described in more detail with reference to the accompanying drawings. While some examples are shown in the drawings, it should be understood that solutions can be implemented in various forms and should not be construed as limited to the examples presented herein. Rather, these examples are provided to provide a more thorough and complete understanding of the solutions. It should be understood that the drawings and examples in this document are for illustrative purposes only and are not intended to limit the scope of protection of the solutions.

[0011] It should be noted that the headings of any section or subsection provided in this disclosure are not limiting. Various examples are described throughout this document, and examples of any type may be included under any section or subsection. Furthermore, examples described in any section or subsection may be combined in any way with any other examples described in the same or different sections or subsections.

[0012] In the description of examples in this disclosure, the term "comprising" and similar terms should be understood as open inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "an example" or "the example" should be understood as "at least one example". The term "some examples" should be understood as "at least some examples". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0013] The examples in this article may involve user data, data acquisition, and usage. All of these aspects comply with relevant laws, regulations, and rules. In the examples, all data collection, acquisition, processing, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, when implementing each example, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained through appropriate means, in accordance with relevant laws and regulations. The specific methods of notification and authorization can vary depending on the actual situation and application scenario; the scope of the solution is not limited in this regard.

[0014] In this manual and its examples, any processing of personal information will be conducted only on a legal basis (e.g., with the consent of the data subject, or as necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect their use of basic functions.

[0015] As used in this article, the term "multi-core processor" refers to a processor with multiple processing cores capable of performing computational tasks. Each processing core may have independent or partially shared storage resources, data transport resources, and computational constraints.

[0016] As used in this paper, the term "computation graph" refers to a graph structure used to represent computational tasks and the data dependencies between them. A computation graph can be a directed acyclic graph (DAG) or other graph structures that can be processed by a compiler.

[0017] The term "task unit" as used in this paper refers to a processing unit in a computation graph that can be used as a task assignment object. A task unit may include a single operator or a set of operators formed according to data dependencies.

[0018] As used in this paper, the term "operator" refers to a computational operation that performs a predetermined computation on input data and produces output data. Operators can include ordinary convolution, depthwise convolution, fully connected layers, matrix multiplication, transpose, data rearrangement, element-wise operations, group convolution, or group normalization.

[0019] As used in this article, "resource configuration information" refers to information describing the processing resources and computational constraints available for performing computational tasks in a multi-core processor. Resource configuration information may include the set of processing cores, the default number of cores, the default partition axis, the default partition granularity, storage capacity, data transport capabilities, and alignment constraints.

[0020] The term "computational constraint" as used in this article refers to the hardware conditions that processing resources must meet when performing computational tasks. Computational constraints can involve data precision, tensor dimensions, channel alignment, grouping relationships, accessible storage range, and data transfer capabilities between processing cores.

[0021] The term "task allocation information" as used in this article refers to information indicating how computational tasks are partitioned and the processing resources used to execute the partitioned computational tasks. Task allocation information may include the partition axis, partition granularity, number of cores, and set of processing cores.

[0022] The term "historical task allocation information" as used in this article refers to the task allocation information recorded for task units prior to the current task allocation. Historical task allocation information can originate from a previous allocation of the same computation graph, or from an allocation prior to a modification of the computation graph.

[0023] The term "splitting axis" as used in this article refers to the dimension in a tensor used to partition the computational task. The splitting axis can be a channel dimension, batch dimension, sequence dimension, matrix row dimension, matrix column dimension, or other dimensions determined by dimension mapping relationships.

[0024] The term "segmentation granularity" as used in this paper refers to the number of data units contained in a basic task group formed along the segmentation axis. For convolution computation, data units can be input channels or output channels.

[0025] The term “number of cores” as used in this article refers to the number of processing cores that are assigned to at least one basic task group and actually participate in the corresponding computational tasks.

[0026] The term "natural partition axis" as used in this paper refers to the preferred partition dimension determined based on the operator type and its tensor semantics. The natural partition axis is used to form the initial task assignment information for task units.

[0027] As used in this paper, the term "data dependency" refers to a relationship where the input of one task unit depends on the output of another task unit. This relationship can be represented by edges in a computation graph, data transfer operations, or shared data objects.

[0028] The term "tensor" as used in this article refers to a data set with one or more dimensions. Tensor properties can include shape, data precision, data layout, and storage location.

[0029] The term "alignment constraint" as used in this article refers to a divisibility or multiple condition imposed by processing resources on the amount or storage arrangement of data in a basic task group. Different operator types, data precisions, and processing core sets can correspond to different alignment constraints.

[0030] The term "basic task group" as used in this article refers to the basic allocation unit that is divided from the computational tasks of the task unit according to the granularity of the division. A processing core can execute one or more basic task groups.

[0031] The term "forward traversal" as used in this paper refers to processing task units sequentially along the data transfer direction of the computation graph, so that the upstream task allocation information affects the traversal process of the downstream task units.

[0032] The term "backward traversal" as used in this paper refers to processing task units sequentially in the reverse direction of data transmission, so that downstream computational constraints affect the traversal process of upstream task units.

[0033] The term "multi-input branch" as used in this paper refers to a graph structure in which multiple upstream data paths converge to the same target operator. Multiple input branches can correspond to element-wise addition, element-wise multiplication, or affine transformation.

[0034] The term “grouping computation constraint” used in this article refers to the channel divisibility relation, input-output grouping relation, and hardware alignment condition that must be satisfied when dividing input and output channels into multiple independent processing groups.

[0035] The term "equivalent transformation operator" as used in this paper refers to an operator that changes the available granularity of data segmentation or the arrangement of data while maintaining consistency between the data before and after processing. Equivalent transformation operators can include identity convolution.

[0036] The term “core utilization” as used in this article refers to the ratio between the number of processing cores actually involved in a computing task and the number of processing cores available for that computing task.

[0037] As used in this document, the term "Direct Memory Access (DMA)" refers to the method by which data is transferred between storage locations or processing cores by data transport resources. Its availability is indicated by resource configuration information.

[0038] The term “Least Common Multiple (LCM)” used in this paper refers to the smallest positive integer that can be divided by multiple positive integers at the same time. It can be used to find candidate segmentation granularities that satisfy multiple channel grouping and alignment conditions.

[0039] In some embodiments, each processing core of a multi-core processor has its own independent local memory. One processing core cannot directly read local data from another processing core, or can only do so with the aid of constrained data transport resources. In this execution environment, downstream task units need to obtain upstream output data corresponding to their task partitions. Otherwise, even if each task unit can complete its local computation independently, the computation graph may still not execute continuously due to cross-core data inaccessibility.

[0040] A compiler can execute multiple compilation optimization processes sequentially on the same computation graph. A compilation optimization process may insert at least one of the following: data transfer operators, format conversion operators, equivalent transformation operators, tensor shapes, data layouts, storage locations, or data precision. The aforementioned modifications will change the graph structure, data relationships, or alignment conditions that were in place when the original task allocation information was valid.

[0041] Therefore, the computational task allocation process needs to be able to be re-executed for modified computation graphs. Re-execution must eliminate the influence of historical information and ensure that adjacent task units, multiple input branches, operations that change dimensional arrangement, and grouping operators receive mutually compatible task allocation information. This process also needs to maintain consistent data dependencies when different data precisions correspond to different processing core sets and alignment constraints.

[0042] A computational task allocation scheme for multi-core processors is proposed. According to this scheme, the electronic device acquires the computation graph and resource configuration information, and clears historical task allocation information when initiating computational task allocation. Based on the current computation graph and resource constraints, the electronic device generates initial task allocation information for each task unit, and then propagates and coordinates this information along data dependencies. The electronic device thus obtains mutually compatible target task allocation information and allocates the computational tasks of each task unit to multiple processing cores.

[0043] Through the above scheme, electronic devices can reconstruct task allocation results corresponding to the current state after changes in the computation graph or computational constraints. Task allocation information propagates along data dependency paths, enabling downstream task units to obtain the data blocks required for their computation and ensuring that data blocks at the convergence of multiple inputs remain consistent. Multiple processing cores can thus execute computational tasks according to the coordinated task partitioning, reducing invalid computations and redundant data transfers caused by conflicting partitioning information.

[0044] The following describes various examples of this scheme in further detail with reference to the accompanying drawings.

[0045] Figure 1 A schematic diagram of example environment 100 is shown.

[0046] like Figure 1 As shown, environment 100 includes processing device 110, computation graph 120, and target task allocation information 130. Processing device 110 may include a multi-core processor, memory, and a program for executing the compilation process. Computation graph 120 may be stored in the memory of processing device 110 or provided by a storage device communicatively connected to processing device 110. Processing device 110 processes computation graph 120 and generates target task allocation information 130.

[0047] The nodes in computation graph 120 can represent task units, and the directed edges between nodes can represent the data dependencies of the output tensor from upstream task units to downstream task units. Computation graph 120 may include concatenation paths, multi-input branches, multi-output branches, and branch merging paths. Processing device 110 can traverse these nodes according to the topological relationships and update the task allocation information according to the type of operator and resource constraints corresponding to each node.

[0048] The target task allocation information 130 can record the partition axis, partition granularity, number of cores, and set of processing cores for each task unit. This information can be used by subsequent processing by the compiler to generate instructions or configurations for executing each computational task on a multi-core processor.

[0049] In some embodiments, the processing cores of a multi-core processor cannot directly access each other's local data. In other embodiments, direct memory access resources can aggregate data from multiple processing cores to a single processing core, or send data from one processing core to multiple processing cores. The processing device 110 can record the actual data transfer capabilities in resource configuration information, making subsequent task allocation suitable for both strictly isolated local storage structures and multi-core structures with constrained aggregation or transmission capabilities.

[0050] It should be understood that the structure and function of the various elements in environment 100 are described for illustrative purposes only and do not imply any limitation on the scope of the scheme.

[0051] The following description of the example will continue with reference to the accompanying drawings.

[0052] Figure 2 A flowchart illustrating an example of a process 200 for allocating computational tasks for a multi-core processor according to some embodiments of the present disclosure is shown. Reference is made below. Figure 2 To describe process 200.

[0053] In step 210, the processing device 110 acquires the computation graph and the resource configuration information of the multi-core processor. The computation graph includes multiple task units, each task unit including one or more operators and corresponding to a computation task to be executed by the multi-core processor. The resource configuration information is used to indicate the processing resources available in the multi-core processor for executing the computation task and the computational constraints of the processing resources.

[0054] The computation graph can be computation graph 120, and can use a directed acyclic graph to organize task units and data dependency edges, or other graph structures that can be processed by the compiler. The computation graph can include concatenated paths, multi-input branches, multi-output branches, and branch merging paths. Resource configuration information can be provided by the memory of processing device 110. The task unit is the basic object for subsequent initial analysis and information propagation, and processing resources can include multiple sets of processing cores, data transport resources, and storage resources.

[0055] Resource configuration information can record the set of available processing cores according to processing core type and data precision. For example, integer calculations can correspond to a general-purpose processing core set, while floating-point calculations can correspond to a processing core set with specific computing capabilities. Resource configuration information can also record the default number of cores, default split axis, default split granularity, channel alignment value, local storage capacity, and cross-core data transfer capability. In the following examples, the number of cores, channels, alignment value, and number of traversal rounds are all unrestricted values, and each value is determined by the actual hardware configuration.

[0056] In this way, processing device 110 incorporates the computation graph structure and the actual resource constraints of the current multi-core processor into a single solution. Subsequent steps can then generate task partitioning that can be executed by the target hardware, taking into account the available core set, storage capacity, and alignment conditions.

[0057] In step 220, in response to initiating computational task allocation for the computation graph, processing device 110 clears historical task allocation information for multiple task units. Processing device 110 may delete the partition axis, partition granularity, and core count recorded for each task unit in the previous computational task allocation. Processing device 110 may also reset these attributes to the default partition axis, default partition granularity, and default core count indicated by the resource configuration information, so that the current computational task allocation is independent of the results of the previous computational task allocation.

[0058] Historical task allocation information represents the computational state left over from the previous processing, with default values ​​provided by the resource configuration information as the starting point for resolving the problem. Clearing does not delete task units or operators in the computation graph; instead, it removes task partitioning attributes that may be inconsistent with the current graph structure.

[0059] In some embodiments, the processing device 110 can detect whether other compilation and optimization processes have modified the computation graph. This modification may include inserting data transfer operators, format conversion operators, or equivalent transformation operators, and may also include changing at least one of the following: tensor shape, data layout, storage location, or data precision. Insertion operators alter the upstream and downstream tensor transfer relationships, and changes in data layout or data precision may cause the original segmentation granularity to no longer meet the new alignment requirements. Upon detecting any modification, the processing device 110 can restart process 200 and clear the task allocation information prior to the modification in step 220.

[0060] In this way, each execution of process 200 starts from the state corresponding to the current resource configuration. The compiler does not need to maintain the invalidated partition state between multiple optimization iterations. Processing device 110 can also avoid generating tasks based on the old partition axis or the old number of cores, thereby reducing recompilation and invalid task scheduling caused by changes in the graph structure.

[0061] In step 230, the processing device 110 determines the initial task allocation information for each of the multiple task units based on the computation graph and resource configuration information. The task allocation information indicates the division method of the computation tasks and the processing resources used to execute the divided computation tasks. The target task allocation information may record the partitioning axis, partitioning granularity, number of cores, and set of processing cores.

[0062] The initial task allocation information represents the local scheme obtained by each task unit before it has coordinated with adjacent task units. The processing device 110 can first determine the natural splitting axis based on the operator type, then select the target processing core set and alignment constraints based on the data precision, and then determine the initial splitting granularity and the number of initial cores based on the output tensor shape and alignment constraints.

[0063] Operator types may include ordinary convolution, depthwise convolution, fully connected operations, and matrix multiplication. For convolution operators with output channels as the natural splitting axis, processing device 110 can determine the number of basic task groups using the ratio between the number of output channels and the initial splitting granularity. Processing device 110 assigns the basic task groups to available processing cores in the target processing core set and determines the number of processing cores assigned to at least one basic task group as the initial core number.

[0064] If the basic task groups cannot be evenly distributed, the processing device 110 can adjust the initial partitioning granularity according to the values ​​allowed by the alignment constraints. The adjusted granularity still satisfies the hardware alignment conditions and makes the number of basic task groups undertaken by each processing core more similar.

[0065] As an example of a regular convolution, a task unit may include a 3×3 convolution. The input tensor shape can be (1, 32, 56, 56). The number of output channels can be 64. The data precision can be an 8-bit integer (INT8). The processing device 110 can first reset the splitting granularity to 4, the number of cores to 8, and the splitting axis to the channel dimension. After recognizing the convolution and data precision, the processing device 110 can determine that the channel alignment value is 8. The 64 output channels are divided into 8 basic task groups according to a granularity of 8, and each basic task group is assigned to a processing core, thus the initial splitting granularity is 8 and the initial number of cores is 8.

[0066] The output of the ordinary convolution can be passed to downstream task units via data transport resources. The downstream task units inherit the splitting granularity of 8 and the number of cores of 8; if the downstream task unit is connected to a multi-input operation, the processing device 110 continues to perform multi-input branch coordination. In this way, the local initialization scheme of the ordinary convolution can serve as the deterministic input for subsequent propagation.

[0067] As an extension of step 230 when adjacent task units use different data precisions, the processing device 110 can also perform precision constraint coordination between local initial schemes and then send the coordination results to step 240.

[0068] In some embodiments, the tensors processed by adjacent first task units and second task units may have different data precisions. The processing device 110 may determine the maximum value of the segmentation granularity of the two task units as the target segmentation granularity, determine the minimum value of the number of cores of the two task units as the target number of cores, and update the task allocation information of at least one task unit based on the target segmentation granularity and the target number of cores.

[0069] Different data precisions can correspond to different processing core sets and different alignment values. Data precision can include 32-bit integers, 8-bit integers, and 16-bit floating-point numbers (FP16). For example, integer calculations can use a general-purpose processing core set, while floating-point calculations can use a processing core set with corresponding floating-point capabilities. A larger granularity of partitioning ensures that both task units meet their respective alignment conditions, while a smaller number of cores ensures that the number of data blocks on both sides of the precision conversion position corresponds.

[0070] Based on the initial analysis above, ordinary convolution, depthwise convolution, fully connected layers, and matrix multiplication can form initial partitions according to their respective tensor semantics. Basic task groups are only assigned to processing cores that actually participate in computation, avoiding the inclusion of processing cores without computational tasks in the core count. Data precision participates in the selection of core sets and alignment values, ensuring that subsequent propagation does not require repeatedly trying combinations that do not meet hardware conditions at precision change points, nor does it require manually specifying the processing core set and partitioning configuration related to data precision.

[0071] In step 240, the processing device 110 propagates task allocation information among multiple task units according to the data dependencies of the computation graph. The task allocation information of task units that have data dependencies with the determined task units is updated based on the task allocation information of the determined task units, so that the task allocation information of task units with data dependencies are compatible with each other, and the target task allocation information of each of the multiple task units is obtained.

[0072] Propagation refers to the transmission of the partition axis, partition granularity, and number of cores along the data dependency edges. Compatibility means that the data blocks formed by the data producer can be obtained by the data consumer, and adjacent task units satisfy their respective alignment, grouping, and data precision constraints.

[0073] If the input to the current task unit comes from external storage, the processing device 110 can retain the initial task allocation information of the current task unit. If the current task unit receives the output tensor of an upstream task unit through a data transfer operation, the processing device 110 can update the current task unit based on the upstream task allocation information. The processing device 110 can also update the upstream task units connected to the downstream task units based on the downstream task allocation information.

[0074] For example, when an upstream task unit divides the output channel into multiple basic task groups and assigns them to multiple processing cores, a downstream task unit can obtain the output tensor along the same data block boundary. For structures where local storage cannot be directly accessed across cores, the corresponding data block of the downstream task unit is arranged at the processing core that can obtain that data block. For data transfer structures that support aggregation or transmission, the processing device 110 can decide whether to transfer data through direct memory access resources based on resource configuration information.

[0075] Processing device 110 can perform a forward traversal along the data transfer direction of the computation graph to propagate upstream task allocation information to downstream task units. Processing device 110 can also perform a backward traversal in the opposite direction to propagate downstream constraints to upstream task units. Processing device 110 repeats the forward and backward traversals until the task allocation information no longer changes or the number of traversal rounds reaches a preset number of rounds.

[0076] Forward and backward traversals can be performed alternately. For example, processing device 110 can perform two forward traversals and two backward traversals. The number of rounds can be determined by resource configuration information or compilation configuration, and is not limited to four rounds. Updates from the previous round form the input for the next round, allowing incompatibilities found at multiple input merging points, grouping operators, or precision changes to have a reverse impact on earlier task units.

[0077] Through bidirectional propagation, the entry task unit can retain the initial scheme formed based on external input, while the internal task unit can inherit the upstream data partitioning and accept downstream constraints. Functionally, each data path in the computation graph receives a continuous task partitioning. At the machine execution level, mutually compatible data blocks reduce task failures caused by cross-core unreachability and also reduce the number of times data is repeatedly generated and moved to fix local conflicts.

[0078] In one example implementation, the computation graph may include a target operator with multiple input branches. Processing device 110 searches upstream along each of the multiple input branches for task units that have data dependencies on the target operator, and collects the granularity of the split, the number of cores, and the split axis for each input branch. Processing device 110 determines the maximum value among the granularities of the input branches as the unified granularity and the minimum value among the number of cores in each input branch as the unified number of cores. Processing device 110 also coordinates the split axes of each input branch based on operators that change the tensor dimension arrangement, and writes the unified granularity, the unified number of cores, and the coordinated split axes back to each input branch.

[0079] Multiple input branches can incorporate element-wise addition, element-wise multiplication, or affine transformations. The maximum granularity of the split is used to simultaneously satisfy the existing granularity and alignment conditions of each branch, while the minimum number of cores ensures that the number of data blocks in the merged branches is consistent. Split axis coordination is used to handle situations where the dimensional order changes due to transposition or data rearrangement in a branch.

[0080] Alternatively or additionally, some embodiments of this paper may also employ two ordinary convolutional branches to illustrate the write-back process for unified information. The left branch convolution outputs 64 channels with a split granularity of 8 and uses 8 processing cores; the right branch convolution outputs 32 channels with a split granularity of 8 and uses 4 processing cores. The two branches are used to illustrate the core number coordination before multiple inputs are merged; when element-wise operations are actually performed, the shape of the corresponding input tensor also needs to conform to the input conditions of the operator.

[0081] Processing device 110 collects the granularity of the cut and the number of cores 8 along the left branch, and collects the granularity of the cut and the number of cores 4 along the right branch. Processing device 110 determines the uniform granularity of the cut and the uniform number of cores to be 8, and keeps the cut axis unchanged after confirming that both branches are cut along the channel dimension.

[0082] When writing back unified information, the right branch maintains 4 processing cores, while the left branch is adjusted from 8 processing cores to 4. In the left branch, each processing core is responsible for 16 output channels instead of 8, and each core handles 2 basic task groups, while the granularity of the split remains 8. This results in two branches forming corresponding allocation positions at the 4-processor-core level at the multi-input convergence point, while retaining the granularity that satisfies the alignment constraints on both sides.

[0083] Figures 3A to 3C Flowcharts of example processes 300A to 300C for task allocation coordination according to some embodiments of the present disclosure are shown. Figure 3A A flowchart of an example process 300A, in which multiple input branches are not yet coordinated, is shown according to some embodiments of this disclosure. Figure 3AAs shown, processing device 110 acquires output information in step 304. For example, processing device 110 can acquire output information from two input branches. In the first branch, processing device 110 acquires a first two-dimensional convolution 306. Then, in step 310, processing device 110 performs bias addition on the first two-dimensional convolution 306 to obtain a first output tensor 312. In the second branch, processing device 110 acquires a second two-dimensional convolution 316. Then, in step 320, processing device 110 performs bias addition on the second two-dimensional convolution 316, and in step 322, performs activation calculation on the second two-dimensional convolution 316 to obtain a second output tensor 324. The shapes of both output tensors are (1, 16, 256, 256). The weight shape of the second two-dimensional convolution is (16, x, 1, 1). Where x represents the number of input channels.

[0084] In step 314, the processing device 110 determines that the first branch has 4 first cores. In step 326, the processing device 110 determines that the second branch has 1 second core. The first branch can correspond to a non-1×1 convolution with a smaller alignment value, and the second branch can correspond to a 1×1 convolution that requires alignment of 16 output channels. The 16 output channels of the former can be allocated to 4 processing cores, and the 16 output channels of the latter form a basic task group and are executed by 1 processing core.

[0085] In step 330, processing device 110 determines that the number of the first cores and the number of the second cores are not equal. The two branches form different numbers of data blocks at the merging point, making direct element-wise addition impossible. In step 332, processing device 110 outputs a first result indicating a matching failure. This failure triggers coordination of task allocation information between the two branches.

[0086] pass Figure 3A The detection path, processing device 110 identifies branch core number conflicts before performing element-by-element operations, avoiding directly handing over data blocks that cannot be matched one-to-one to the element-by-element operator, and providing... Figure 3B or Figure 3C The subsequent coordination provides the triggering conditions.

[0087] Figure 3B A flowchart of an example process 300B for achieving coordination by reducing the number of cores according to some embodiments of this disclosure is shown. (Continued) Figure 3A In step 336, the processing device 110 compares the number of cores and in step 338 determines the uniform number of cores based on the smaller value.

[0088] In step 340, processing device 110 adjusts the number of first cores to 1. In step 342, processing device 110 maintains the number of second cores at 1. The data in the first branch, originally handled by 4 processing cores, is reorganized to be handled by 1 processing core; the second branch does not need to be changed. In step 344, processing device 110 outputs the second result of successful matching, and in step 346, performs the first element-wise addition.

[0089] pass Figure 3B In this method, two branches can form data blocks of the same number and corresponding positions at the merging point, thus completing element-wise computation. This method does not increase the number of operators in the computation graph and is suitable for situations where processing time and storage constraints are still met after reducing the number of cores. At the machine execution level, unifying the number of cores eliminates cross-core correspondence conflicts between branches, but the number of parallel processing cores available to the first branch is correspondingly reduced.

[0090] Figure 3C A flowchart of an example process 300C for achieving coordination by inserting identity convolution according to some embodiments of the present disclosure is shown. (Continued) Figure 3A Following the second output tensor 324 of the second branch, in step 348, the processing device 110 inserts an identity 2D convolution. The weight shape of the identity 2D convolution can be (16, 16, 3, 3). The identity 2D convolution maintains consistency between the tensor data before and after processing.

[0091] In step 348, processing device 110 obtains a third output tensor 352. In step 354, processing device 110 determines the third core count to be 4. Identity 2D convolution allows the second branch to form 4 basic task groups according to a smaller channel alignment value suitable for 3×3 convolution. In step 356, processing device 110 determines that the first and third core counts are equal, outputs the successfully matched third result in step 358, and performs a second element-wise addition in step 360.

[0092] pass Figure 3C In this manner, the second branch expands from one processing core to four processing cores while maintaining consistent output data, corresponding to the number of data blocks recovered by the first branch. Element-by-element addition can thus continue, while reducing the serial processing bottleneck caused by alignment constraints in the second branch.

[0093] Figure 3B and Figure 3C This indicates two alternative coordinated implementations. Processing device 110 can first evaluate the number of cores obtained without modifying the computation graph, or it can evaluate the number of cores that can be obtained after inserting an equivalent transformation operator that maintains data consistency. Processing device 110 selects one of the implementations based on resource constraints and core utilization, and does not require the two implementations to be executed simultaneously.

[0094] Alternatively or additionally, the processing device 110 identifies a target operation in the computation graph that alters the arrangement of tensor dimensions and determines the mapping relationship between the input and output dimensions of the target operation. The target operation may include at least one of transpose, data rearrangement, sequence-to-batch format conversion, and data redistribution. Based on the mapping relationship, the processing device 110 maps the upstream partitioning axis of the target operation to the downstream partitioning axis of the target operation and updates the downstream task unit using the mapped partitioning axis.

[0095] For example, a three-dimensional tensor can be partitioned along axis 2. After the transpose operation swaps axis 1 and axis 2, the data semantics of the original axis 2 correspond to axis 1 of the output tensor. The processing device 110 then updates the partitioning axis of the downstream task unit to axis 1.

[0096] In another example, an M-row, N-column matrix can be divided into four data blocks along the column dimension N. When subsequent normalization calculations require obtaining the complete N-dimensional data at the same processing core, the processing device 110 can adjust the partition axis to the row dimension M based on the dimension mapping and the computational constraint. This adjustment does not change the matrix elements, but rather changes the direction in which the computational task is partitioned between processing cores.

[0097] Through dimension mapping, the data semantics expressed by the segmented information can continue to be transmitted across transpose and data rearrangement operations. Functionally, downstream task units obtain the segmentation axis that matches its dimension order, without the need for manual annotation or hard-coding of axis changes. At the machine execution level, the code generation process can directly use the updated dimensions, reducing recompilation and data rearrangement caused by generating tasks along incorrect dimensions.

[0098] Alternatively or additionally, the computation graph may include target operators with grouped computation constraints. Grouped computation constraints mean that the input and output channels are divided into groups of multiple independent processes. Each processing core can handle a complete group, or a portion of a group if the condition of uniform partitioning within the group is met, but cannot simultaneously handle a complete group and an incomplete portion of another group. The number of segments formed on the input and output sides must be the same or an integer multiple thereof.

[0099] Processing device 110 determines candidate segmentation granularities on the input side and output side based on the ratio of the number of input channels to the number of input groups, the ratio of the number of output channels to the number of output groups, alignment constraints indicated by resource configuration information, and the segmentation granularity propagated to the target operator. Processing device 110 verifies whether the number of input channels and the number of output channels are divisible by the corresponding candidate segmentation granularities, and verifies that the number of input-side segments is the same as or an integer multiple of the number of output-side segments. Processing device 110 selects the grouping task allocation information from the verified candidate segmentation granularities that maximizes the number of processing cores used by the target operator.

[0100] As an example, the number of input and output channels in a grouped convolution can both be 128. The input and output are divided into 4 groups, each containing 32 channels. The tensor can be a 16-bit floating-point number and executed by 4 corresponding processing cores. The least common multiple of 32 and 4 is 32, so the candidate splitting granularity on both the input and output sides can be determined as 32. The 128 channels are divided into 4 groups with a granularity of 32, and both the input and output sides can be assigned to 4 processing cores.

[0101] Processing device 110 confirms that 128 is divisible by 32, and confirms that the number of input side-cut groups and the number of output side-cut groups are both 4. Therefore, this candidate scheme satisfies the channel divisibility relation, group correspondence relation, and alignment constraint. The specific values ​​in this example are used to illustrate the solution process; other channel numbers, group numbers, data precision, and number of processing cores can also be solved according to the same constraints.

[0102] If the task allocation information propagated to the grouping operator does not satisfy the channel divisibility relation, group correspondence relation, or alignment constraint, the processing device 110 does not use the candidate scheme and returns the incompatible constraint to the upstream task unit. The processing device 110 then adjusts the upstream segmentation information and re-executes the forward and backward traversals until a valid candidate scheme is formed or the preset number of rounds is reached.

[0103] Through grouping constraint solving, the input and output data blocks of the objective operator maintain consistent group boundaries. Functionally, grouped convolution and grouped normalization can participate in the overall graph task allocation without disrupting independent processing groups. At the machine execution level, the processing device 110 selects the scheme with the larger number of actual processing cores from the legal candidates, enabling the available processing cores to undertake the computational task and avoiding additional data splicing caused by incomplete grouping.

[0104] As an extension of the aforementioned group constraint solution under low core utilization conditions, the processing device 110 can evaluate whether to change the available splitting conditions through equivalent transformation while maintaining the group boundary constraints.

[0105] The processing device 110 can also determine the first number of cores used by the target operator based on the grouped task allocation information. If the core utilization determined based on the first number of cores is lower than the core utilization threshold, the processing device 110 can determine the second partitioning granularity and the second number of cores corresponding to inserting the equivalent transformation operator on the input side of the target operator. The equivalent transformation operator adjusts the available partitioning granularity while maintaining data consistency before and after processing. If the second number of cores is greater than the first number of cores, the processing device 110 inserts the equivalent transformation operator in the computation graph and repropagates the task allocation information for the adjusted computation graph.

[0106] As an example, the output channels of a grouped convolution can be 16, divided into 4 groups, each containing 4 channels. The number of available processing cores can be 4, and the current partitioning granularity can be 16. Directly partitioning at a granularity of 16 forms one basic task group, thus requiring only one processing core to handle the computation, resulting in a core utilization rate of 25%.

[0107] Processing device 110 can evaluate inserting a 3×3 identity convolution when the number of output channels is no greater than a threshold (e.g., 64). Identity convolution allows the use of an alignment value of 4, thus the 16 output channels can be grouped into four basic task groups at a granularity of 4 and assigned to four processing cores, achieving 100% core utilization. Since the number of second cores (4) is greater than the number of first cores (1), processing device 110 inserts an identity convolution, ensuring that the inputs of the original grouped convolutions are first processed through the identity convolution. After the insertion operation is complete, processing device 110 returns to the task allocation information propagation process, enabling upstream and downstream task units to re-coordinate based on the modified computation graph.

[0108] Identical convolution is just one example of an equivalent transformation operator. Resource configuration information can specify other operators that can maintain data consistency, allowed insertion positions, and core utilization thresholds. Processing device 110 only modifies the computation graph when the number of processing cores expected to be used after insertion increases, thereby avoiding the introduction of additional computation that does not improve the degree of parallel execution in order to adjust the splitting conditions.

[0109] Through equivalent transformation evaluation, processing device 110 can change the available splitting conditions of the grouping operator while maintaining tensor data consistency. Functionally, the multi-input merge location or grouping operator location can obtain task allocation information compatible with the number of cores. At the machine execution level, the actual number of cores used in the example increases from 1 to 4, and the number of processing cores without allocated computing tasks decreases accordingly.

[0110] The aforementioned identity convolution insertion modifies the computation graph, and other compilation optimizations may also generate the same type of reentrancy requirement. Therefore, the modified graph structure needs to be reconnected to the processing links of steps 220 to 240.

[0111] If grouping, data transfer, format conversion, or other compilation optimizations modify the computation graph, processing device 110 restarts the computation task allocation. Processing device 110 clears the task allocation information before the modification and redetermines the target task allocation information for the modified computation graph.

[0112] This restart path connects initialization, initial splitting, cross-task unit propagation, and grouping processing into a repetitive multi-stage process. When grouping processing modifies the computation graph, the process flow returns to the information propagation stage or re-executes from the initialization stage. Processing device 110 thus avoids retaining out-of-sync local patching results within grouping processing, reducing implementation failures caused by inconsistent stage states.

[0113] In step 250, the processing device 110 allocates the computing tasks of each of the multiple task units to multiple processing cores of the multi-core processor based on the target task allocation information.

[0114] The processing device 110 can form basic task groups according to the segmentation axis and segmentation granularity recorded in each task unit, and then allocate the basic task groups to specific processing cores according to the number of cores and the set of processing cores. When data transfer is required, the processing device 110 can also configure corresponding data transfer operations according to the resource configuration information.

[0115] Through the above scheme, the computational task allocation results correspond to the data dependencies of the current computation graph and the computational constraints of the current multi-core processor. Functionally, ordinary convolution, multi-input operations, dimensionality transformation operations, and grouping operators in the computation graph can be executed continuously using the same set of task allocation information. At the machine execution level, each processing core obtains the data block corresponding to its own task, reducing idle processing cores, failed tasks, and duplicate data transfer caused by partitioning conflicts. Even after modifications to the computation graph or during the compilation stage, repropagation can maintain consistency of task allocation information between different processing stages.

[0116] A corresponding apparatus for implementing the above methods or processes is also provided.

[0117] Figure 4 A schematic structural block diagram of an apparatus 400 illustrating examples of computational task allocation for a multi-core processor according to some embodiments of the present disclosure is shown. Apparatus 400 may be implemented as or included in processing device 110. Various modules or components in apparatus 400 may be implemented by hardware, software, firmware, or any combination thereof.

[0118] like Figure 4As shown, the apparatus 400 includes an acquisition module 410, a clearing module 420, a determination module 430, a propagation module 440, and an allocation module 450. The acquisition module 410 is configured to acquire a computation graph and resource configuration information of a multi-core processor. The computation graph includes multiple task units, each task unit including one or more operators and corresponding to a computation task to be executed by the multi-core processor. The resource configuration information indicates the processing resources available for executing the computation task in the multi-core processor and the computational constraints of those resources. The clearing module 420 is configured to clear historical task allocation information of multiple task units in response to initiating computation task allocation on the computation graph. The determination module 430 is configured to determine initial task allocation information for each of the multiple task units based on the computation graph and the resource configuration information. The task allocation information indicates the partitioning method of the computation task and the processing resources used to execute the partitioned computation task. The propagation module 440 is configured to propagate the task allocation information among the multiple task units according to the data dependencies of the computation graph. Specifically, the task allocation information of task units with data dependencies on the already determined task units is updated based on the task allocation information of the already determined task units, so that the task allocation information of task units with data dependencies is compatible with each other, and the target task allocation information of each of the multiple task units is obtained. The allocation module 450 is configured to allocate the computing tasks of each of the multiple task units to multiple processing cores of the multi-core processor based on the target task allocation information.

[0119] In some embodiments, the clearing module 420 can also be configured to delete the partition axis, partition granularity, and core count recorded for each task unit in the previous computation task allocation. The clearing module 420 can also be configured to reset the partition axis, partition granularity, and core count of each task unit to the default partition axis, default partition granularity, and default core count indicated by the resource configuration information, so that the current computation task allocation does not depend on the result of the previous computation task allocation.

[0120] In some embodiments, the determining module 430 may further be configured to determine the natural splitting axis of the task unit based on the ordinary convolution, depthwise convolution, fully connected, or matrix multiplication operators included in the task unit. The determining module 430 may also be configured to select a target processing core set from multiple processing core sets based on the data precision of the tensor processed by the task unit, and determine alignment constraints corresponding to the data precision. The determining module 430 may further be configured to determine the initial splitting granularity and the initial number of cores of the task unit based on the shape of the output tensor and the alignment constraints.

[0121] In some embodiments, the determining module 430 may further be configured to determine the number of basic task groups based on the ratio of the size of the output tensor on the natural partition axis to the initial partition granularity. The determining module 430 may also be configured to evenly distribute the basic task groups to available processing cores in the target processing core set, and determine the number of processing cores allocated to at least one basic task group as the initial core number. The determining module 430 may further be configured to adjust the initial partition granularity according to the values ​​allowed by the alignment constraints when the basic task groups cannot be evenly distributed, in order to balance the computational load of each processing core.

[0122] In some embodiments, the propagation module 440 can also be configured to retain the initial task allocation information of the current task unit when the input of the current task unit comes from storage space outside the current task unit. The propagation module 440 can also be configured to update the task allocation information of the current task unit based on the task allocation information of the upstream task unit when the current task unit receives the output tensor of the upstream task unit through a data transmission operation. The propagation module 440 can also be configured to update the task allocation information of the upstream task unit connected to the downstream task unit based on the task allocation information of the downstream task unit.

[0123] In some embodiments, the propagation module 440 can also be configured to perform a forward traversal of multiple task units along the data transmission direction of the computation graph to propagate the task allocation information of upstream task units to downstream task units. The propagation module 440 can also be configured to perform a backward traversal of multiple task units in the opposite direction of the data transmission direction to propagate the constraints of downstream task units to upstream task units. The propagation module 440 can also be configured to repeat the forward and backward traversals until the task allocation information of the multiple task units no longer changes or the number of traversal rounds reaches a preset number of rounds.

[0124] In some embodiments, the propagation module 440 can also be configured to update the task allocation information of task units adjacent to the target operator when the computation graph includes a target operator with multiple input branches. The propagation module 440 can also be configured to search upstream along the multiple input branches for task units that have data dependencies on the target operator, collecting the granularity of each input branch, the number of cores, and the granularity of the granularity. The propagation module 440 can also be configured to determine the maximum value among the granularities of each input branch as the unified granularity, and the minimum value among the number of cores in each input branch as the unified number of cores. The propagation module 440 can also be configured to coordinate the granularity of each input branch based on operators that change the tensor dimension arrangement in each input branch, and write the unified granularity, the unified number of cores, and the coordinated granularity back to each input branch.

[0125] In some embodiments, the propagation module 440 may also be configured to identify target operations that change the tensor dimension arrangement in the computation graph, and determine the mapping relationship between the input and output dimensions of the target operation. The target operation includes at least one of transpose, data rearrangement, sequence-to-batch format conversion, and data redistribution. The propagation module 440 may also be configured to map the upstream partitioning axis of the target operation to the downstream partitioning axis of the target operation based on the mapping relationship, and update the task allocation information of the downstream task unit using the mapped partitioning axis.

[0126] In some embodiments, the propagation module 440 can also be configured to, when the computation graph includes a target operator with grouping computation constraints, determine candidate input-side and candidate output-side segmentation granularities based on the ratio of the number of input channels to the number of input groups, the ratio of the number of output channels to the number of output groups, the alignment constraints indicated by the resource configuration information, and the segmentation granularity propagated to the target operator. The grouping computation constraints are used to indicate that the input and output channels of the target operator are divided into multiple independently processed groups. The propagation module 440 can also be configured to verify whether the number of input channels and the number of output channels are divisible by the corresponding candidate segmentation granularities, and to verify that the number of input-side segmented groups is the same as or an integer multiple of the number of output-side segmented groups. The propagation module 440 can also be configured to, among the verified candidate segmentation granularities, determine the grouping task allocation information that maximizes the number of processing cores used by the target operator.

[0127] In some embodiments, the propagation module 440 can also be configured to determine a first number of cores used by the target operator based on the grouped task allocation information. The propagation module 440 can also be configured to determine a second partitioning granularity and a second number of cores corresponding to inserting an equivalent transformation operator on the input side of the target operator when the core utilization determined based on the first number of cores is lower than a core utilization threshold. The equivalent transformation operator is used to adjust the available partitioning granularity while maintaining data consistency before and after processing. The propagation module 440 can also be configured to insert an equivalent transformation operator in the computation graph when the second number of cores is greater than the first number of cores, and to redistribute the task allocation information among multiple task units for the adjusted computation graph.

[0128] In some embodiments, the clearing module 420 and the propagation module 440 may also be configured to detect whether other compilation optimization processes have modified the computation graph. Modifications may include at least one of the following: inserting data transfer operators, format conversion operators, equivalent transformation operators, changing tensor shape, data layout, storage location, and data precision. The clearing module 420 and the propagation module 440 may also be configured to, in response to detecting a modification, restart computation task allocation, clear the task allocation information before the modification, and redetermine the target task allocation information for the modified computation graph.

[0129] In some embodiments, the determining module 430 may further be configured to determine the maximum value among the segmentation granularities of the first task unit and the second task unit as the target segmentation granularity when the tensors processed by adjacent first task units and second task units have different data precisions. The determining module 430 may further be configured to determine the minimum value among the core counts of the first task unit and the second task unit as the target core count, and update the task allocation information of at least one of the first task unit and the second task unit based on the target segmentation granularity and the target core count.

[0130] The modules included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more modules can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 400 can be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), and so on.

[0131] Figure 5 A block diagram of an electronic device 500 in which one or more embodiments may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the examples described herein. Figure 5 The electronic device 500 shown can be used to implement the computing task allocation device 400 for multi-core processors as discussed above.

[0132] like Figure 5As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processing units or processors 510, memory 520, storage devices 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processor 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processors execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0133] Electronic device 500 typically includes multiple computer storage media. Such media can be any accessible media that is accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof). Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data and accessible within electronic device 500.

[0134] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various examples.

[0135] The communication unit 540 is used to enable communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device 500 can be implemented using a single computing cluster or multiple computers capable of communicating via communication connections. Therefore, the electronic device 500 can operate in a networked environment using logical connections to one or more other servers, networked personal computers, or another network node.

[0136] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. External devices may include storage devices, display devices, devices for enabling user interaction with electronic device 500, or devices for enabling electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interfaces (not shown).

[0137] A computer-readable storage medium is provided that stores computer-executable instructions thereon, wherein the computer-executable instructions are executed by a processor to implement the method described above. The computer-readable storage medium can be disposed in an electronic device or communicatively connected to an electronic device.

[0138] A computer program product is also provided, comprising computer-executable instructions. When executed by a processor, the computer-executable instructions cause the processor to implement the methods or processes described above. The computer program product can run on or be implemented in an electronic device, or it can be run collaboratively by the electronic device and a server. The electronic device can implement the methods described above, either alone or in collaboration with a server, through the computer program product. The computer program product may include, for example, applications running on the device, functional modules, application plug-in software installation packages, firmware upgrade packages, client programs, or cloud programs. For example, as described above, the computer program product may include a computing task allocation application, a computing task allocation application, a computing graph compilation application, a computing task allocation functional module, an application plug-in, a client program, or a cloud-based computing task allocation program for multi-core processors. The electronic device can implement the methods or processes described in the embodiments of this disclosure, for example, by running the computer program product. For example, by running computer program products, electronic devices can obtain the resource configuration information of the computation graph and multi-core processors, clear the historical task allocation information of multiple task units, and determine the initial task allocation information of each of the multiple task units; electronic devices can propagate the task allocation information among multiple task units according to the data dependencies of the computation graph, obtain the target task allocation information, and allocate the computing tasks of each of the multiple task units to multiple processing cores based on the target task allocation information.

[0139] A computer program product is tangibly stored on a non-transient computer-readable medium and includes computer-executable instructions, and is installed, loaded, or invoked into an electronic device, wherein the computer-executable instructions are executed by the processor of the electronic device to implement the methods described above.

[0140] The flowcharts and / or block diagrams of the methods, apparatus, devices, and computer program products referred to herein describe various aspects. It should be understood that each block of the flowcharts and / or block diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0141] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0142] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to some embodiments. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0144] Various examples have been described above. The foregoing descriptions are exemplary and not exhaustive, nor are they limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used in this disclosure is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for allocating computing tasks for a multi-core processor, characterized in that, include: The computation graph and the resource configuration information of the multi-core processor are obtained. The computation graph includes multiple task units, each task unit includes one or more operators and corresponds to a computation task to be executed by the multi-core processor. The resource configuration information is used to indicate the processing resources in the multi-core processor that can be used to execute the computation task and the computation constraints of the processing resources. In response to initiating computation task allocation for the computation graph, the historical task allocation information of the multiple task units is cleared; Based on the computation graph and the resource configuration information, the initial task allocation information of each of the plurality of task units is determined. The task allocation information is used to indicate the division method of the computation tasks and the processing resources used to execute the divided computation tasks. According to the data dependencies of the computation graph, task allocation information is propagated among the multiple task units. The task allocation information of the task units that have data dependencies with the determined task units is updated based on the task allocation information of the determined task units, so that the task allocation information of the task units with data dependencies are compatible with each other, and the target task allocation information of each of the multiple task units is obtained. as well as Based on the target task allocation information, the computing tasks of each of the multiple task units are allocated to multiple processing cores of the multi-core processor.

2. The method according to claim 1, characterized in that, The process of clearing the historical task allocation information of the multiple task units includes: Delete the partition axis, partition granularity, and core count recorded in the previous computation task allocation for each task unit; and The partitioning axis, partitioning granularity, and core count of each task unit are reset to the default partitioning axis, default partitioning granularity, and default core count indicated by the resource configuration information, so that the current computation task allocation does not depend on the result of the previous computation task allocation.

3. The method according to claim 1, characterized in that, Determining the initial task allocation information for each of the plurality of task units includes: The natural splitting axis of the task unit is determined based on the ordinary convolution, depthwise convolution, fully connected or matrix multiplication operators contained in the task unit. Based on the data precision of the tensors processed by the task unit, a target processing core set is selected from multiple processing core sets, and alignment constraints corresponding to the data precision are determined; and Based on the output tensor shape of the task unit and the alignment constraints, the initial segmentation granularity and the initial number of cores of the task unit are determined.

4. The method according to claim 3, characterized in that, Determining the initial number of cores for the task unit includes: The number of basic task groups is determined based on the ratio of the size of the output tensor on the natural slicing axis to the initial slicing granularity. If the basic task groups can be evenly distributed, the basic task groups are evenly distributed to the available processing cores in the target processing core set, and the number of processing cores distributed to at least one basic task group is determined as the initial number of cores; and If the basic task groups cannot be evenly distributed, the initial partitioning granularity is adjusted according to the values ​​allowed by the alignment constraints to balance the computational load of each processing core.

5. The method according to claim 1, characterized in that, The process of propagating task allocation information among the plurality of task units includes: If the input to the current task unit comes from storage space outside the current task unit, retain the initial task allocation information of the current task unit; When the current task unit receives the output tensor of the upstream task unit through a data transmission operation, the task allocation information of the current task unit is updated based on the task allocation information of the upstream task unit; and Based on the task allocation information of the downstream task unit, update the task allocation information of the upstream task unit connected to the downstream task unit.

6. The method according to claim 5, characterized in that, The process of propagating task allocation information among the plurality of task units includes: The multiple task units are traversed forward along the data transmission direction of the computation graph to propagate the task allocation information of the upstream task units to the downstream task units. The plurality of task units are traversed backwards in the opposite direction of the data transmission direction to propagate the constraints of downstream task units to upstream task units; and Repeat the forward traversal and the backward traversal until the task allocation information of the multiple task units no longer changes or the number of traversal rounds reaches the preset number of rounds.

7. The method according to claim 1, characterized in that, The computation graph includes an objective operator with multiple input branches, and the method further includes: Search upstream along the multiple input branches for task units that have data dependencies on the target operator, and collect the granularity of each input branch, the number of cores, and the splitting axis. The maximum value among the segmentation granularities of each input branch is determined as the unified segmentation granularity, and the minimum value among the core counts of each input branch is determined as the unified core count; and Based on the operators that change the tensor dimension arrangement in each input branch, the splitting axis of each input branch is coordinated, and the unified splitting granularity, the unified core number, and the coordinated splitting axis are written back to each input branch.

8. The method according to claim 7, characterized in that, The process of propagating task allocation information among the plurality of task units further includes: The computation graph identifies target operations that alter the arrangement of tensor dimensions and determines the mapping relationship between the input and output dimensions of the target operations. The target operations include at least one of transpose, data rearrangement, sequence-to-batch format conversion, and data redistribution. Based on the mapping relationship, the upstream splitting axis of the target operation is mapped to the downstream splitting axis of the target operation, and the task allocation information of the downstream task unit is updated using the mapped splitting axis.

9. The method according to claim 1, characterized in that, When the computation graph includes an objective operator with grouped computation constraints, the method further includes: Based on the ratio of the number of input channels to the number of input groups, the ratio of the number of output channels to the number of output groups of the target operator, the alignment constraints indicated by the resource configuration information, and the segmentation granularity propagated to the target operator, candidate segmentation granularities on the input side and candidate segmentation granularities on the output side are determined respectively. The grouping calculation constraint is used to indicate that the input channels and output channels of the target operator are divided into multiple independently processed groups. Verify whether the number of input channels and the number of output channels are divisible by the corresponding candidate segmentation granularity, and verify that the number of input side-slicing groups is the same as or an integer multiple of the number of output side-slicing groups; and Among the candidate segmentation granularities that pass the verification, the grouping task allocation information that maximizes the number of processing cores used by the target operator is determined.

10. The method according to claim 9, characterized in that, Also includes: The number of first cores used by the target operator is determined based on the group task allocation information; If the core utilization rate determined based on the first core number is lower than the core utilization rate threshold, a second segmentation granularity and a second core number are determined when an equivalent transformation operator is inserted on the input side of the target operator, wherein the equivalent transformation operator is used to adjust the available segmentation granularity while keeping the data consistent before and after processing. as well as When the number of the second cores is greater than the number of the first cores, the equivalent transformation operator is inserted into the computation graph, and the task allocation information is redistributed among the multiple task units for the adjusted computation graph.

11. The method according to claim 1 or 2, characterized in that, Also includes: The system detects whether other compilation optimization processes modify the computation graph. The modifications include at least one of the following: inserting data transfer operators, format conversion operators, equivalent transformation operators, changing tensor shape, data layout, storage location, and data precision. as well as In response to the detection of the modification, the computation task allocation is restarted, the task allocation information before the modification is cleared, and the target task allocation information is re-determined for the modified computation graph.

12. The method according to claim 1, characterized in that, Also includes: When the tensors processed by adjacent first task units and second task units have different data precision, the maximum value of the segmentation granularity of the first task unit and the second task unit is determined as the target segmentation granularity, the minimum value of the number of cores of the first task unit and the second task unit is determined as the target number of cores, and the task allocation information of at least one of the first task unit and the second task unit is updated based on the target segmentation granularity and the target number of cores.

13. A computing task allocation device for a multi-core processor, characterized in that, The device includes: The acquisition module is configured to acquire a computation graph and resource configuration information of the multi-core processor. The computation graph includes multiple task units, each task unit includes one or more operators and corresponds to a computation task to be executed by the multi-core processor. The resource configuration information is used to indicate the processing resources available for executing the computation task in the multi-core processor and the computational constraints of the processing resources. The clearing module is configured to clear the historical task allocation information of the plurality of task units in response to initiating computation task allocation for the computation graph. The determination module is configured to determine the initial task allocation information of each of the plurality of task units based on the computation graph and the resource configuration information. The task allocation information is used to indicate the division method of the computation tasks and the processing resources used to execute the divided computation tasks. A propagation module is configured to propagate task allocation information among the plurality of task units according to the data dependencies of the computation graph, wherein the task allocation information of task units with data dependencies on the determined task units is updated based on the task allocation information of the determined task units to ensure that the task allocation information of the data-dependent task units is compatible with each other, and to obtain the target task allocation information for each of the plurality of task units; and The allocation module is configured to allocate the computing tasks of each of the multiple task units to multiple processing cores of the multi-core processor based on the target task allocation information.

14. An electronic device, characterized in that, include: At least one processor; as well as At least one memory coupled to the at least one processor and storing instructions for execution by the at least one processor, the instructions causing the electronic device to perform the method according to any one of claims 1 to 12 when executed by the at least one processor.

15. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that can be executed by a processor to implement the method according to any one of claims 1 to 12.

16. A computer program product, characterized in that, The computer program product is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 1 to 12.

Images