Task allocation method and device for AI chip, chip, equipment and storage medium

Abstract

The embodiment of the invention discloses a task allocation method and device for an AI chip, the chip, equipment and a storage medium, and belongs to the technical field of AI. The method comprises the steps that a to-be-executed task of an AI chip is determined, the AI chip comprises a first execution unit and a second execution unit, the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port; determining a first communication task from the tasks to be executed, wherein the first communication task indicates to send first data in the AI chip to the outside of the AI chip; distributing the first communication task to a first execution unit; wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data to the outside of the AI chip. AXI identifier conflicts in the AXI communication process are avoided, and resources are saved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence (AI) technology, and more specifically, to a task allocation method, apparatus, chip, device, and storage medium for an AI chip. Background Art

[0002] AI chip multi-card interconnection refers to connecting multiple AI chips (such as GPUs) so that they can work together to improve computing performance and efficiency and meet the needs of large-scale AI tasks.

[0003] In AI chip communication scenarios involving multiple interconnected cards, the Advanced eXtensible Interface (AXI) is a commonly used standard bus protocol, typically used for point-to-point (P2P) communication between multiple cards. AXI identifiers (AXI IDs) are primarily used to identify and manage transactions, ensuring the correctness and reliability of data transmission. Summary of the Invention

[0004] The present invention proposes a task allocation method, apparatus, chip, device, and storage medium for an AI chip, which helps to mitigate or overcome conflicts during communication.

[0005] The technical solutions of the embodiments of the present invention are as follows: A task allocation method for an AI chip, the method comprising: Determining a task to be executed by an AI chip, the AI chip comprising a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier or any communication port; Determining a first communication task from the tasks to be executed, where the first communication task instructs sending first data in the AI chip to an outside of the AI chip; assigning the first communication task to the first execution unit; The first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip.

[0006] In one embodiment, it further includes: determining a non-communication task from the tasks to be performed; assigning the non-communication task to the second execution unit; The second execution unit executes the non-communication task.

[0007] In one embodiment, allocating the non-communication task to the second execution unit includes: Selecting an execution unit with the largest amount of available computing resources from among all execution units that are not associated with any AXI identifier and any communication port; The execution unit with the largest amount of available computing resources is determined as the second execution unit.

[0008] In one embodiment, it further includes: Determining a second communication task from the tasks to be executed, where the second communication task instructs receiving second data from outside the AI chip; assigning the second communication task to the first execution unit; The first execution unit executes the second communication task based on the first communication port to receive the second data from outside the AI chip.

[0009] In one embodiment, the AI chip includes a third execution unit, the third execution unit is associated with a second AXI identifier and a second communication port in the AI chip, and the method includes: Determining a second communication task from the tasks to be executed, where the second communication task instructs receiving second data from outside the AI chip; assigning the second communication task to the third execution unit; The third execution unit executes the second communication task based on the second communication port to receive the second data from outside the AI chip.

[0010] In one embodiment, the first execution unit is an execution unit in the AI chip having the smallest communication delay with the first communication port; or The first execution unit is an execution unit with the best communication performance in the AI chip, which is determined based on a communication performance monitoring tool or historical communication records.

[0011] A task allocation device for an AI chip, the device comprising: a first determining module, configured to determine a task to be executed by an AI chip, wherein the AI chip includes a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier or any communication port; a second determining module, configured to determine a first communication task from the tasks to be executed, where the first communication task instructs to send first data in the AI chip to an outside of the AI chip; An allocation module is configured to allocate the first communication task to the first execution unit; wherein the first execution unit performs the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip.

[0012] An AI chip, comprising: a first execution unit, associated with a first AXI identifier and a first communication port in the AI chip; The second execution unit is not associated with any AXI identifier and any communication port; a scheduling unit configured to determine tasks to be executed by the AI chip; determine a first communication task from the tasks to be executed, the first communication task instructing to send first data in the AI chip to an outside of the AI chip; and assign the first communication task to the first execution unit; The first execution unit is configured to execute the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip.

[0013] In one embodiment, the scheduling unit is configured to determine a non-communication task from the tasks to be executed; and assign the non-communication task to the second execution unit; The second execution unit is configured to execute the non-communication task.

[0014] An electronic device, comprising: Memory; processor; The memory stores an application program executable by the processor, which is used to enable the processor to execute any of the above-described task allocation methods for AI chips.

[0015] A computer-readable storage medium having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by a processor, the processor executes the task allocation method for an AI chip as described in any one of the above items.

[0016] A program product includes a computer program, which, when executed by a processor, implements the task allocation method for an AI chip as described in any one of the above items.

[0017] As can be seen from the above technical solution, in an embodiment of the present invention, the AI chip includes a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier or any communication port; a first communication task is determined from the tasks to be executed, and the first communication task indicates that the first data in the AI chip is to be sent outside the AI chip; the first communication task is assigned to the first execution unit; wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip. Thus, it can be seen that the first execution unit sends the data in the transaction of the AXI identifier associated with itself based on the communication port associated with itself, avoiding AXI identifier conflicts during the communication process and saving resources.

[0018] In addition, allocating other types of tasks that do not belong to the first communication task to other execution units different from the first execution unit can improve the sending efficiency of the first execution unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of a dual-SIM communication scenario in the related art.

[0020] Figure 2 Schematic diagram of the communication status of a communication port in the related art.

[0021] Figure 3 4 is an exemplary flowchart of a task allocation method for an AI chip according to an embodiment of the present invention.

[0022] Figure 4 Schematic diagram of a dual-SIM communication scenario according to an embodiment of the present invention.

[0023] Figure 5 Schematic diagram of performing full protocol operation based on dual-SIM communication according to an embodiment of the present invention.

[0024] Figure 6 2 is an exemplary structural diagram of a task allocation device for an AI chip according to an embodiment of the present invention.

[0025] Figure 7 is an exemplary structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings.

[0027] For the sake of brevity and intuitiveness in description, the solution of the present invention is explained below by describing several representative implementations. A large number of details in the implementations are only used to help understand the solution of the present invention. However, it is obvious that the technical solution of the present invention may not be limited to these details when implemented. In order to avoid unnecessarily obscuring the solution of the present invention, some implementations are not described in detail, but only a framework is given. Hereinafter, "including" means "including but not limited to", and "according to..." means "at least according to..., but not limited to only according to...". Due to the language habits of Chinese, when the number of a component is not specifically specified below, it means that the component can be one or more, or can be understood as at least one.

[0028] The AXI protocol identifier (AXI identifier) uniquely distinguishes transactions. In multi-master and high-concurrency scenarios, AXI identifiers play a critical role in maintaining transaction ordering and improving system performance. An AXI transaction is a complete data transfer operation on the AXI bus, typically consisting of a transaction request and a transaction response. Each AXI transaction (e.g., read request, write request, read response, and write response) carries an AXI identifier (with a configurable bit width, such as 4 or 8 bits) that uniquely identifies the transaction initiator or transaction group. The AXI protocol allows for out-of-order completion of transactions, but transactions with the same AXI identifier must complete in chronological order. This means that if a master sends multiple transaction requests with the same AXI identifier, slaves or the interconnect must return transaction responses in the exact order in which they were initiated.

[0029] In the current hardware architecture of AI chips, each execution unit in the AI chip is associated with its own communication port, which may lead to communication conflicts and waste of resources. For example, the execution unit can be implemented as a stream processing cluster (SPC) unit.

[0030] The following describes the dual-SIM communication scenario as an example. Figure 1 Schematic diagram of dual-SIM card communication scenario in related technology. Figure 1Middle: SPC unit 0 (labeled as spc0) and SPC unit 4 (labeled as spc4) in GPU0 are associated with P2P communication port 0 (labeled as P2P port 0) in GPU0; SPC unit 1 (labeled as spc1) and SPC unit 5 (labeled as spc5) in GPU0 are associated with P2P communication port 1 (labeled as P2P port 1) in GPU0; SPC unit 2 (labeled as spc2) and SPC unit 6 (labeled as spc6) in GPU0 are associated with P2P communication port 2 (labeled as P2P port 2) in GPU0; SPC unit 3 (labeled as spc3) and SPC unit 7 (labeled as spc7) in GPU0 are associated with P2P communication port 3 (labeled as P2P port 3) in GPU0.

[0031] Assume that the maximum number of allocatable AXI identifiers is 4, namely AXI identifier 0, AXI identifier 1, AXI identifier 2, and AXI identifier 3. Transactions with AXI identifier 0 are marked in green, transactions with AXI identifier 1 are marked in yellow, transactions with AXI identifier 2 are marked in blue, and transactions with AXI identifier 3 are marked in red.

[0032] Spc0 on GPU0 requests to send data in a transaction with AXI identifier 0. This data is then sent to GPU1 via P2P port 0, which is associated with spc0. While P2P port 0 is sending data from spc0, data in a transaction with AXI identifier 0 requested by spc2 is blocked. Spc2 data can only be sent via P2P port 1 after P2P port 0 has finished sending data from spc0. However, this waiting period wastes resources.

[0033] Figure 2 FIG. 1 is a schematic diagram of the communication status of a communication port in the related art. Figure 2 In the figure, the direction of the arrow is time (T). Figure 2 It can be seen that P2P port 2 starts sending data in the transaction with AXI identifier 0 of spc2 only after P2P port 0 has finished sending data in the transaction with AXI identifier 0 of spc2. During the waiting period, the resources of P2P port 2 are in an idle state.

[0034] After research, the inventors found that the reasons for the above-mentioned waste of resources include at least the following: in the related art, each execution unit is associated with its own communication port, the communication port is not bound to the AXI identifier, and each execution unit has communication capabilities. However, based on the AXI protocol, requests for the same AXI identifier need to be completed in chronological order. Therefore, at different communication ports, it may be necessary to send data with the same AXI identifier provided by different execution units at the same time (which can be called an AXI identifier conflict). At this time, even if there is an idle communication port in the AI chip, the data that appears later needs to wait until the data that appears earlier with the same AXI identifier is sent before it can be sent, which leads to a waste of resources.

[0035] In an embodiment of the present invention, a number of specific execution units (for example, equal to the number of communication ports) are selected from all execution units of the AI chip, and each specific execution unit is associated with its own communication port and associated with its own AXI identifier. Each specific execution unit sends the data in the transaction of the AXI identifier associated with itself based on the communication port to which it is associated, thereby avoiding AXI identifier conflicts during the AXI communication process. The remaining execution units in the AI chip, except for the specific execution units, are not associated with the communication port or the AXI identifier, and do not have the ability to send data in the AXI identifier transaction. In addition, other types of tasks that do not belong to the sending task are assigned to these remaining execution units, thereby ensuring the sending efficiency of the specific execution units.

[0036] The above disclosure details the technical defects in the related art, the causes of these defects, and the analytical process for overcoming them. In reality, the understanding of these technical defects is not common knowledge in the field, but rather a novel discovery made by the inventors during their research. Furthermore, the tracing of the causes of these defects and the analytical process for overcoming them are the result of gradual analysis conducted by the inventors during their actual research and are not common knowledge in the field.

[0037] Figure 3 4 is an exemplary flowchart of a task allocation method for an AI chip according to an embodiment of the present invention. Figure 3The method shown can be executed by a processor in an AI chip. For example, the AI chip can be implemented as any one of: a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), and a general-purpose graphics processing unit (GPGPU). Figure 3 As shown, the method includes: Step 101: Determine a task to be executed of an AI chip, where the AI chip includes a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port.

[0038] The first execution unit can send data in a transaction with its associated first AXI identifier to the outside of the AI chip based on its associated first communication port. The second execution unit is neither associated with a communication port nor an AXI identifier and does not send data to the outside of the AI chip.

[0039] For example, the input task queue of the AI chip can be continuously monitored to determine the tasks to be executed by the AI chip. For example, the tasks to be executed may include: (1) Computational tasks: For example, computing tasks can include deep learning inference tasks, deep learning training tasks, and machine learning tasks.

[0040] Deep learning reasoning tasks can include: (a) neural network forward propagation tasks: performing forward calculations of the neural network on the input data to obtain the output results; (b) model deployment and reasoning tasks: deploying the trained deep learning model on the chip and performing reasoning calculations on new data. For example, in natural language processing, reasoning on text data to generate text summaries or translation results.

[0041] Deep learning training tasks can include: (a) backpropagation and gradient calculation: calculating the gradient of the loss function with respect to the network weights for model optimization and update. For example, when training an image classification model, the gradient is calculated to adjust the network parameters based on the difference between the predicted results and the true labels; (b) parameter update and optimization: using the calculated gradient, the network parameters are updated through optimization algorithms (such as SGD, Adam, etc.) to improve the performance of the model.

[0042] Machine learning tasks can include: (a) feature extraction and transformation: extracting and transforming features from raw data to improve the performance and efficiency of the model. For example, in the data preprocessing stage, data can be standardized, normalized, and subjected to principal component analysis. (b) model training and prediction: training traditional machine learning models (such as support vector machines, decision trees, random forests, etc.) and using the trained models to predict and classify new data.

[0043] (2) Data processing tasks: For example, data processing tasks may include data preprocessing and data postprocessing.

[0044] Data preprocessing can include: (a) data cleaning and denoising: removing noise and outliers from the data, filling or interpolating missing data to improve data quality; (b) data normalization and standardization: normalizing or standardizing the data to meet specific distribution or range requirements.

[0045] Data post-processing can include: (a) result organization and format conversion: organizing and formatting the calculation or inference results to make them meet the requirements of subsequent processing or output, such as converting the output results of deep learning models into readable text or image formats; (b) data storage and management: storing and managing the processed data for subsequent query and use, such as storing the data in the chip cache or external storage device.

[0046] (3) Communication tasks: For example, communication tasks include: (a) data transmission tasks: sending data generated within the AI chip (such as calculation results, model parameters, etc.) to other chips or devices through communication ports (usually P2P communication ports). For example, in multi-GPU parallel computing, some calculation results are sent to other GPUs for further processing; (b) data reception tasks: receiving data from other chips or devices through P2P communication ports. For example, in distributed training, this involves receiving gradient information or model parameter updates sent by other nodes.

[0047] The above exemplary descriptions are specific methods for determining tasks to be performed and typical examples of tasks to be performed. Those skilled in the art will appreciate that such descriptions are merely exemplary and are not intended to limit the scope of protection of the embodiments of the present invention.

[0048] Step 102: Determine a first communication task from tasks to be executed, where the first communication task instructs to send first data in the AI chip to outside the AI chip.

[0049] Here, a first communication task is determined from the pending tasks to instruct the transmission of first data in the AI chip to a location outside the AI chip. The first data may be any data in the AI chip. For example, the first data may include gradient information, updated model parameters or parameter update amounts, intermediate calculation results, or final settlement results.

[0050] Step 103: Allocate the first communication task to a first execution unit in the AI chip, wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip.

[0051] An AI chip can contain multiple execution units capable of sending data outside the AI chip. Each execution unit capable of sending data outside the AI chip is associated with its own AXI identifier and communication port. Each execution unit capable of sending data outside the AI chip also has the ability to receive data from outside the AI chip.

[0052] For example, assigning the first communication task to the first execution unit includes: selecting the first execution unit from all execution units that have the ability to send data outside the AI chip based on the idleness. For example, the execution unit with the largest idleness is determined as the first execution unit. In an optional embodiment, the first execution unit is selected from all execution units that have the ability to send data outside the AI chip based on the communication delay. For example, the execution unit with the smallest communication delay with the first communication port in the AI chip is determined as the first execution unit. In an optional embodiment, the first execution unit is the execution unit with the best communication performance in the AI chip, determined based on a communication performance monitoring tool or historical communication records.

[0053] By assigning the first communication task to a first execution unit associated with the first communication port and the first AXI identifier, the first execution unit can execute the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip. Specifically, the process of sending the first data may include: first, sending the write address and write control information (for example, the write control information may include the first AXI identifier) outside the AI chip (for example, to another AI chip) through the first communication port; then, sending the first data carrying the first AXI identifier through the first communication port. Because the first communication port that sends the first data is associated with the first AXI identifier, while the other communication ports are not associated with the first AXI identifier, when the first communication port sends data with the first AXI identifier, there is no need for the other communication ports to send data with the first AXI identifier, thereby avoiding identifier conflicts and overcoming or alleviating the problem of resource waste.

[0054] In one embodiment, the method further includes: determining a non-communication task (e.g., a computing task or data processing task, etc.) from the tasks to be executed; assigning the non-communication task to a second execution unit; wherein the second execution unit executes the non-communication task. Assigning the non-communication task to the second execution unit (rather than also assigning it to the first execution unit) can improve the transmission efficiency of the first execution unit. Here, non-communication tasks refer to tasks other than communication tasks for transmitting data between AI chips. Communication tasks may include sending data within an AI chip to other chips via a P2P communication port, or receiving data from other chips via a P2P communication port. For example, non-communication tasks may specifically include tasks that require reading high-bandwidth memory (HBM) payloads (e.g., reading data and then performing summation).

[0055] In one embodiment, assigning the non-communication task to the second execution unit includes: selecting the second execution unit based on available computing resources from all execution units not associated with any AXI identifier and any communication port. For example, the execution unit with the largest amount of available computing resources is selected as the second execution unit.

[0056] Specifically, when selecting the execution unit with the largest amount of available computing resources, the available computing resources may be weighed based on any one or more of the following methods in any combination.

[0057] Method (1): Measure available computing resources based on specific quantitative indicators.

[0058] For example, an execution unit with the largest number of available computing cores or the largest available memory capacity may be selected as the second execution unit.

[0059] Method (2): Measure available computing resources based on computing power assessment.

[0060] For example, the execution unit with the highest computing capability may be selected as the second execution unit.

[0061] Method (3): Available computing resources can be measured based on the estimation of task execution time.

[0062] For example, when executing tasks of the same type, the execution unit with the shortest task completion time is estimated to be the second execution unit.

[0063] Method (4): Dynamically evaluate available computing resources based on the current load.

[0064] For example, the execution unit with the lowest load at the current moment may be selected as the second execution unit.

[0065] In one embodiment, the method further includes: determining a second communication task from the tasks to be executed, the second communication task indicating receiving second data from outside the AI chip; assigning the second communication task to the first execution unit; wherein the first execution unit executes the second communication task based on the first communication port to receive the second data from outside the AI chip.

[0066] Therefore, the first execution unit can perform both the first communication task of sending first data from the AI chip to a location outside the AI chip and the second communication task of receiving second data from a location outside the AI chip. When performing the second communication task, the first execution unit directly receives the second data using the first communication port without associating the second data with the first AXI identifier. In this case, the data receiving task and the data sending task are each assigned to the same execution unit with communication capabilities.

[0067] In one embodiment, an AI chip includes a third execution unit, the third execution unit is associated with a second AXI identifier and a second communication port in the AI chip, and the method includes: determining a second communication task from tasks to be executed, the second communication task indicating receiving second data from outside the AI chip; assigning the second communication task to the third execution unit; wherein the third execution unit executes the second communication task based on the second communication port to receive the second data from outside the AI chip.

[0068] Therefore, embodiments of the present invention can also utilize a third execution unit, different from the first execution unit, to perform a second communication task of receiving second data from outside the AI chip. When performing the second communication task, the third execution unit directly receives the second data using the second communication port without associating the second data with the second AXI identifier. In this case, the data receiving task and the data sending task are respectively assigned to different execution units with communication capabilities.

[0069] Figure 4 Schematic diagram of a dual-SIM communication scenario according to an embodiment of the present invention. Figure 4 Middle: SPC unit 0 (labeled as spc0), SPC unit 1 (labeled as spc1), SPC unit 2 (labeled as spc2), and SPC unit 3 (labeled as spc3) in GPU0 are associated with their respective communication ports and their respective AXI identifiers. Among them: spc0 is associated with P2P communication port 0 (labeled as P2P port 0) and AXI identifier 0, so spc0 has the ability to send data in transactions with AXI identifier 0 based on P2P port 0, and also has the ability to receive data based on P2P port 0; spc1 is associated with P2P communication port 1 (labeled as P2P port 1) and AXI identifier 1, so spc1 has the ability to send data in transactions with AXI identifier 1 based on P2P port 1, and also has the ability to receive data based on P2P port 1; spc2 is associated with P2P communication port 2 (labeled as P2P port 2) and AXI identifier 2, so spc2 has the ability to send data in transactions with AXI identifier 2 based on P2P port 2, and also has the ability to receive data based on P2P port 2; spc3 is associated with P2P communication port 3 (labeled as P2P port 3) and AXI identifier 3, so spc3 has the ability to send data in transactions with AXI identifier 2 based on P2P port 2, and also has the ability to receive data based on P2P port 2; 3 has the ability to send data in transactions with AXI identifier 3, and also has the ability to receive data based on P2P port 3. The remaining execution units in GPU0 (spc4 to spc7) are neither associated with a P2P communication port nor an AXI identifier, and do not have communication capabilities.

[0070] Because spc0-spc3 can each send data in transactions with their associated AXI identifiers based on their associated communication ports, AXI identifier conflicts can be avoided during AXI communication. Furthermore, tasks other than sending tasks can be assigned to spc4-spc7, ensuring the sending efficiency of spc0-spc3.

[0071] The present invention also provides an AI chip. The AI chip includes: a first execution unit associated with a first AXI identifier and a first communication port in the AI chip; a second execution unit not associated with any AXI identifier or any communication port; a scheduling unit configured to determine tasks to be executed by the AI chip; determine a first communication task from the tasks to be executed, the first communication task instructing to send first data in the AI chip to an external device; assign the first communication task to the first execution unit; wherein the first execution unit is configured to execute the first communication task based on the first communication port and the first AXI identifier to send the first data to an external device.

[0072] The scheduling unit can be implemented as a combination of hardware and software. Its core logic can be implemented in hardware circuits to ensure high-speed and efficient task scheduling. At the same time, the scheduling unit can include corresponding software or firmware support to implement more complex scheduling algorithms and strategies, as well as to coordinate with other parts of the chip. For example, the scheduling unit can embed a microprocessor or microcontroller to run the scheduling algorithm and manage the task queue.

[0073] Therefore, by assigning communication and non-communication tasks to different execution units, the chip's task processing efficiency is improved. At the same time, by associating specific communication tasks with specific AXI identifiers and communication ports, the efficiency and reliability of data transmission can be optimized.

[0074] In one embodiment, the scheduling unit is configured to determine a non-communication task from tasks to be executed; and assign the non-communication task to the second execution unit; and the second execution unit is configured to execute the non-communication task.

[0075] The following describes the implementation of the present invention by taking an all-reduce operation as an example. Figure 5 Schematic diagram of performing full protocol operation based on dual-SIM communication according to an embodiment of the present invention.

[0076] exist Figure 5 In this example, GPU0's high-bandwidth memory (HBM) stores data a and data b. Data a is data to be sent to GPU1 for reduction, while data b is data to be reduced by GPU0 itself. GPU1's HBM stores data c and data d. Data c is data to be sent to GPU0 for reduction, while data d is data to be reduced by GPU1 itself.

[0077] spc0 in GPU0 is associated with communication port P2P port0 in GPU0, and spc0 is associated with AXI identifier 0. spc1 in GPU1 is associated with communication port P2P port1 in GPU1, and spc1 is associated with AXI identifier 5.

[0078] In GPU0, the communication task of sending data a to GPU1 is assigned to spc0. spc0 performs this communication task by sending data a to GPU1's P2P port 1 via P2P port 0 using AXI identifier 0. GPU1's spc1 writes data a to GPU1's HBM workspace.

[0079] In GPU1, the communication task of sending data c to GPU0 is assigned to spc1. Spc1 performs this communication task by sending data c to P2P port 0 of GPU0 via P2P port 1 using AXI identifier 5. GPU1's spc0 writes data c to the HBM workspace of GPU0.

[0080] In GPU0, the execution unit with the most available computing resources is spc4. spc4 is assigned the remaining tasks in the full reduction operation except for the communication task, specifically: (1) reading data b from GPU0's HBM and reading data c from GPU0's workspace; (2) calculating the reduction result of data b and data c (assuming it is b + c); and (3) writing the reduction result to GPU0's HBM workspace.

[0081] In GPU1, the execution unit with the most available computing resources is spc5. spc5 is assigned the remaining tasks in the full reduction operation except for the communication task, specifically: (1) reading data d from GPU1's HBM and reading data a from GPU1's workspace; (2) calculating the reduction result of data a and data d (assuming it is a + d); and (3) writing the reduction result to GPU1's HBM workspace.

[0082] As shown in the figure, assigning communication tasks to spc0 and spc1, which are dedicated to communication, avoids AXI identifier conflicts. Furthermore, assigning tasks not related to communication to spc4 and spc5, which do not need to perform communication, also ensures the communication efficiency of spc0 and spc1.

[0083] Figure 6 FIG. 1 is an exemplary structural diagram of a task allocation device for an AI chip according to an embodiment of the present invention. Figure 6As shown, the device includes: a first determination module, used to determine the tasks to be executed of the AI chip, the AI chip includes a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port; a second determination module, used to determine a first communication task from the tasks to be executed, the first communication task indicating that the first data in the AI chip is sent outside the AI chip; an allocation module, used to allocate the first communication task to the first execution unit; wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip.

[0084] In summary, in an embodiment of the present invention, a task to be executed of an AI chip is determined, and the AI chip includes a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port; a first communication task is determined from the tasks to be executed, and the first communication task indicates that the first data in the AI chip is to be sent outside the AI chip; the first communication task is assigned to the first execution unit; wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip. It can be seen that the first execution unit sends the data in the transaction of the AXI identifier associated with itself based on the communication port associated with itself, thereby avoiding AXI identifier conflicts during the AXI communication process and saving resources.

[0085] In addition, allocating other types of tasks that do not belong to the first communication task to other execution units can also ensure the sending efficiency of the first execution unit.

[0086] An embodiment of the present invention further provides an electronic device having a processor-memory architecture. Figure 7 FIG is a structural diagram of an electronic device according to an embodiment of the present invention. Figure 7As shown, the electronic device includes a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the computer program is executed by the processor 801, any of the above-mentioned task allocation methods for AI chips is implemented. Among them, the memory 802 can be specifically implemented as a variety of storage media such as an electrically erasable programmable read-only memory (EEPROM), a flash memory (Flash memory), and a programmable read-only memory (PROM). The processor 801 can be implemented to include one or more central processing units or one or more field programmable gate arrays, wherein the field programmable gate array integrates one or more central processing unit cores. Specifically, the central processing unit or central processing unit core can be implemented as a CPU, GPU, GPGPU, MCU, or DSP, etc.

[0087] Embodiments of the present invention also provide a program product comprising a computer program. When executed by a processor, the computer program implements any of the above-described methods for allocating tasks to an AI chip. The program product may be stored in a computer-readable storage medium. Computer-readable storage media include, but are not limited to, optical disks, hard disks, solid-state drives, USB flash drives, removable hard disks, read-only memory (ROM), random-access memory (RAM), magnetic disk storage, optical disk storage, and the like. These storage media can be connected to a computer system via wired or wireless communication, enabling the computer system to read the computer program product from the storage medium and load it into memory as needed to execute the corresponding functions.

[0088] It should be noted that not all steps and modules in the above processes and structure diagrams are required, and certain steps or modules can be omitted based on actual needs. The execution order of the steps is not fixed and can be adjusted as needed. The division of the modules is merely for the convenience of describing the functional division adopted. In actual implementation, a module can be implemented by multiple modules, and the functions of multiple modules can be implemented by the same module. These modules can be located in the same device or in different devices.

[0089] The hardware modules in each embodiment may be implemented mechanically or electronically. For example, a hardware module may include a specially designed permanent circuit or logic device (e.g., a dedicated processor, such as an FPGA or ASIC) to perform a specific operation. For example, a specific operation can be performed in various types of chips (e.g., AI chips). A hardware module may also include a programmable logic device or circuit (e.g., a general-purpose processor or other programmable processor) temporarily configured by software to perform a specific operation. As for whether to implement the hardware module mechanically, using a dedicated permanent circuit, or using a temporarily configured circuit (e.g., configured by software), the decision can be made based on cost and time considerations.

[0090] The present invention also provides a machine-readable storage medium storing instructions for causing a machine to execute the method described herein. Specifically, a system or device equipped with a storage medium storing software program code that implements the functions of any of the aforementioned embodiments can be provided, and a computer (or CPU or MPU) of the system or device can be configured to read and execute the program code stored in the storage medium. Furthermore, instructions based on the program code can be used to cause an operating system, etc., operating on the computer to perform some or all of the actual operations. The program code read from the storage medium can also be written to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer. Subsequently, based on the program code instructions, a CPU, etc., installed in the expansion board or expansion unit can be configured to perform some or all of the actual operations, thereby implementing the functions of any of the aforementioned embodiments. Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROMs, CD-Rs, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs, and DVD+RWs), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer or cloud via a communication network.

[0091] In this document, "schematic" means "serving as an example, instance or illustration", and any diagram or embodiment described as "schematic" in this document should not be interpreted as a more preferred or more advantageous technical solution. In order to keep the drawings simple, each figure only schematically shows the parts related to the present invention, and does not represent its actual structure as a product. In addition, in order to keep the drawings simple and easy to understand, in some figures, only one of the components with the same structure or function is schematically drawn, or only one of them is marked. In this document, "one" does not mean that the number of relevant parts of the present invention is limited to "only one", and "one" does not mean excluding the situation where the number of relevant parts of the present invention is "more than one". In this document, "upper", "lower", "front", "back", "left", "right", "inside", "outside" and the like are only used to indicate the relative positional relationship between the relevant parts, rather than to limit the absolute positions of these relevant parts.

[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A task allocation method for an AI chip, characterized in that: The method comprises: Determining a task to be executed by an AI chip, the AI chip comprising a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier or any communication port; Determining a first communication task from the tasks to be executed, where the first communication task instructs sending first data in the AI chip to an outside of the AI chip; assigning the first communication task to the first execution unit; The first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip.

2. The method according to claim 1, characterized in that Also includes: determining a non-communication task from the tasks to be performed; assigning the non-communication task to the second execution unit; The second execution unit executes the non-communication task.

3. The method according to claim 2, characterized in that Allocating the non-communication task to the second execution unit comprises: Selecting an execution unit with the largest amount of available computing resources from among all execution units that are not associated with any AXI identifier and any communication port; The execution unit with the largest amount of available computing resources is determined as the second execution unit.

4. The method according to claim 1, wherein Also includes: Determining a second communication task from the tasks to be executed, where the second communication task instructs receiving second data from outside the AI chip; assigning the second communication task to the first execution unit; The first execution unit executes the second communication task based on the first communication port to receive the second data from outside the AI chip.

5. The method according to claim 1, wherein The AI chip includes a third execution unit, the third execution unit is associated with a second AXI identifier and a second communication port in the AI chip, and the method includes: Determining a second communication task from the tasks to be executed, where the second communication task instructs receiving second data from outside the AI chip; assigning the second communication task to the third execution unit; The third execution unit executes the second communication task based on the second communication port to receive the second data from outside the AI chip.

6. The method according to any one of claims 1 to 5, characterized in that The first execution unit is an execution unit in the AI chip having the smallest communication delay with the first communication port; or The first execution unit is an execution unit with the best communication performance in the AI chip, which is determined based on a communication performance monitoring tool or historical communication records.

7. A task allocation device for an AI chip, characterized in that: The device comprises: a first determining module, configured to determine a task to be executed by an AI chip, wherein the AI chip includes a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier or any communication port; a second determining module, configured to determine a first communication task from the tasks to be executed, where the first communication task instructs to send first data in the AI chip to an outside of the AI chip; An allocation module is configured to allocate the first communication task to the first execution unit; wherein the first execution unit performs the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip.

8. An AI chip, characterized in that: include: a first execution unit, associated with a first AXI identifier and a first communication port in the AI chip; The second execution unit is not associated with any AXI identifier and any communication port; a scheduling unit configured to determine tasks to be executed by the AI chip; determine a first communication task from the tasks to be executed, the first communication task instructing to send first data in the AI chip to an outside of the AI chip; and assign the first communication task to the first execution unit; The first execution unit is configured to execute the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip.

9. The AI chip according to claim 8, characterized in that The scheduling unit is configured to determine a non-communication task from the tasks to be executed; and assign the non-communication task to the second execution unit; The second execution unit is configured to execute the non-communication task.

10. An electronic device, characterized in that: include: Memory; processor; The memory stores an application program executable by the processor, which is used to enable the processor to execute the task allocation method for an AI chip according to any one of claims 1 to 6.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, enable the processor to execute the task allocation method for an AI chip according to any one of claims 1 to 6.

12. A program product comprising a computer program, characterized in that When the computer program is executed by a processor, the task allocation method for an AI chip according to any one of claims 1 to 6 is implemented.

Images