Task allocation method and device of AI chip, chip, equipment and storage medium

Abstract

The embodiment of the application discloses a task allocation method and device of an AI chip, a chip, equipment and a storage medium, and belongs to the technical field of AI. The method comprises the following steps: determining a to-be-executed task of an AI chip, the AI chip comprising a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port; determining a first communication task from the to-be-executed task, the first communication task indicating that first data in the AI chip is to be sent out of the AI chip; and allocating the first communication task to the first execution unit; wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier, so as to send the first data out of the AI chip. The method avoids AXI identifier conflicts in the AXI communication process and saves resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence (AI) technology, and more particularly, to a task allocation method and device for an AI chip, a chip, an apparatus, and a storage medium. BACKGROUND

[0002] AI chip multi-card interconnection refers to connecting multiple AI chips (such as GPUs, etc.) to enable them to work collaboratively to improve computing performance and efficiency and meet the needs of large-scale AI tasks.

[0003] In the communication scenario of AI chip multi-card interconnection, the Advanced eXtensible Interface (AXI) is a commonly used standard bus protocol, usually used for point-to-point (P2P) communication between multiple cards. The AXI identifier (AXI ID) is mainly used to identify and manage transactions to ensure the correctness and reliability of data transmission. SUMMARY

[0004] The present application proposes a task allocation method and device for an AI chip, a chip, an apparatus, and a storage medium, which helps to alleviate or overcome conflicts in the communication process.

[0005] The technical solutions of the embodiments of the present application are as follows:

[0006] A task allocation method for an AI chip, the method comprising:

[0007] determining a task to be executed by an AI chip, the AI chip comprising a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port;

[0008] determining a first communication task from the task to be executed, the first communication task indicating that a first data in the AI chip is to be sent out of the AI chip;

[0009] allocating the first communication task to the first execution unit;

[0010] wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data out of the AI chip.

[0011] In one embodiment, further comprising:

[0012] determining a non-communication task from the task to be executed;

[0013] allocating the non-communication task to the second execution unit;

[0014] wherein the second execution unit executes the non-communication task.

[0015] In one embodiment, the assigning the non-communication task to the second execution unit comprises:

[0016] selecting, from all execution units not associated with any AXI identifier and any communication port, an execution unit having a largest amount of available computing resources;

[0017] determining the execution unit having the largest amount of available computing resources as the second execution unit.

[0018] In one embodiment, the method further comprises:

[0019] determining, from the tasks to be executed, a second communication task, the second communication task indicating receiving second data from outside the AI chip;

[0020] assigning the second communication task to the first execution unit;

[0021] wherein the first execution unit executes the second communication task to receive the second data from outside the AI chip based on the first communication port.

[0022] In one embodiment, the AI chip comprises a third execution unit associated with a second AXI identifier and a second communication port in the AI chip, the method comprising:

[0023] determining, from the tasks to be executed, a second communication task, the second communication task indicating receiving second data from outside the AI chip;

[0024] assigning the second communication task to the third execution unit;

[0025] wherein the third execution unit executes the second communication task to receive the second data from outside the AI chip based on the second communication port.

[0026] In one embodiment, the first execution unit is an execution unit in the AI chip having a smallest communication delay with the first communication port; or

[0027] the first execution unit is an execution unit in the AI chip having a best communication performance determined based on a communication performance monitoring tool or historical communication records.

[0028] An apparatus for task allocation of an AI chip, the apparatus comprising:

[0029] The first determining module is configured to determine a to-be-executed task of an AI chip, the AI chip comprising a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port.

[0030] The second determining module is configured to determine a first communication task from the to-be-executed task, the first communication task indicating that first data in the AI chip is to be transmitted out of the AI chip.

[0031] The allocating module is configured to allocate the first communication task to the first execution unit, wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier to transmit the first data out of the AI chip.

[0032] An AI chip comprises:

[0033] A first execution unit associated with a first AXI identifier and a first communication port in the AI chip.

[0034] A second execution unit not associated with any AXI identifier and any communication port.

[0035] A scheduling unit configured to determine a to-be-executed task of an AI chip, determine a first communication task from the to-be-executed task, the first communication task indicating that first data in the AI chip is to be transmitted out of the AI chip, and allocate the first communication task to the first execution unit.

[0036] The first execution unit is configured to execute the first communication task based on the first communication port and the first AXI identifier to transmit the first data out of the AI chip.

[0037] In an embodiment, the scheduling unit is configured to determine a non-communication task from the to-be-executed task, and allocate the non-communication task to the second execution unit.

[0038] The second execution unit is configured to execute the non-communication task.

[0039] An electronic device comprises:

[0040] A memory;

[0041] A processor;

[0042] The memory stores an application executable by the processor, which causes the processor to execute the task allocation method of the AI chip as described in any of the above embodiments.

[0043] A computer readable storage medium having stored thereon computer readable instructions which, when executed by a processor, cause the processor to perform the task allocation method of the AI chip of any one of the above.

[0044] A program product comprising a computer program which, when executed by a processor, implements the task allocation method of the AI chip of any one of the above.

[0045] From the above technical solutions, in the embodiments of the present application, the AI chip comprises a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port; a first communication task is determined from the tasks to be executed, the first communication task indicating that the first data in the AI chip is to be sent out of the AI chip; the first communication task is allocated to the first execution unit; wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data out of the AI chip. As can be seen, the first execution unit sends the data in the transaction of the AXI identifier associated with itself based on the communication port associated with itself, avoiding AXI identifier conflict in the communication process and saving resources.

[0046] In addition, allocating other types of tasks other than the first communication task to other execution units different from the first execution unit can improve the sending efficiency of the first execution unit. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 A schematic diagram of a dual-card communication scenario in the related art.

[0048] Figure 2 A schematic diagram of the communication status of a communication port in the related art.

[0049] Figure 3 An exemplary flowchart of the task allocation method of the AI chip according to the embodiments of the present application.

[0050] Figure 4 A schematic diagram of a dual-card communication scenario according to the embodiments of the present application.

[0051] Figure 5 A schematic diagram of performing full-protocol operation based on dual-card communication according to the embodiments of the present application.

[0052] Figure 6 An exemplary structural diagram of the task allocation device of the AI chip according to the embodiments of the present application.

[0053] Figure 7An exemplary block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the objects, technical solutions, and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings.

[0055] In order to make the objects, technical solutions, and advantages of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings.

[0056] An identifier (AXI identifier) in the AXI protocol is a unique identifier used to distinguish transactions. In multi-master and high concurrency scenarios, the AXI identifier plays a key role in maintaining transaction order and improving system performance. An AXI transaction refers to a complete data transmission operation on an AXI bus, which usually includes a transaction request and a transaction response. Each AXI transaction (such as a read request, a write request, a read response, and a write response) carries an AXI identifier (with a configurable bit width, such as 4 bits or 8 bits, etc.) for uniquely identifying the initiator of the transaction or the transaction group. The AXI protocol allows out-of-order completion of transactions, but transactions with the same AXI identifier need to be completed in time sequence. This means that if a master device sends multiple transaction requests with the same AXI identifier, the slave device or interconnect needs to return transaction responses in strict order according to the initiation order.

[0057] In the hardware architecture of the current AI chip, each execution unit in the AI chip is associated with a respective communication port, which can cause communication conflicts and resource waste problems. For example, the execution unit can be implemented as a Stream Processing Cluster (SPC) unit.

[0058] Take a dual-card communication scenario as an example for illustration. Figure 1 An exemplary diagram of a dual-card communication scenario in the related art. In Figure 1In GPU0, SPC unit 0 (labeled as spc0) and SPC unit 4 (labeled as spc4) are associated with P2P communication port 0 (labeled as P2P port 0) in GPU0; SPC unit 1 (labeled as spcl) and SPC unit 5 (labeled as spc5) are associated with P2P communication port 1 (labeled as P2P port 1) in GPU0; SPC unit 2 (labeled as spc2) and SPC unit 6 (labeled as spc6) are associated with P2P communication port 2 (labeled as P2P port 2) in GPU0; SPC unit 3 (labeled as spc3) and SPC unit 7 (labeled as spc7) are associated with P2P communication port 3 (labeled as P2P port 3) in GPU0.

[0059] Suppose the maximum assignable AXI identifiers are 4, which are AXI identifier 0, AXI identifier 1, AXI identifier 2 and AXI identifier 3, where the transactions of AXI identifier 0 are identified in green, the transactions of AXI identifier 1 are identified in yellow, the transactions of AXI identifier 2 are identified in blue, and the transactions of AXI identifier 3 are identified in red.

[0060] spc0 in GPU0 requests to send data in the transaction of AXI identifier 0, which is sent to GPU1 via P2P port 0 associated with spc0. During the sending of spc0's data by P2P port 0, the data in the transaction of AXI identifier 0 requested to be sent by spc2 will be blocked. After waiting for the sending of spc0's data by P2P port 0 to be finished, the data of spc2 can be sent via P2P port 1. However, this waiting results in resource waste.

[0061] Figure 2 A diagram is shown for the communication status of the communication ports in the related art. In Figure 2 , the arrow direction is time (T). It can be seen from Figure 2 that after waiting for the sending of spc0's data in the transaction of AXI identifier 0 by P2P port 0 to be finished, P2P port 2 starts to send the data in the transaction of AXI identifier 0 of spc2. During the waiting period, the resource of P2P port 2 is in an idle state.

[0062] The inventors have found that the cause of the above-mentioned resource waste at least includes that in the related art, each execution unit is associated with a respective communication port, the communication port is not bound to an AXI identifier, and each execution unit has a communication capability. However, based on the AXI protocol, requests with the same AXI identifier need to be completed in time sequence. Therefore, at different communication ports, data provided by different execution units and having the same AXI identifier (which can be referred to as AXI identifier conflict) can need to be sent at the same time. At this time, even if there is a free communication port in the AI chip, the data appearing later needs to wait until the data appearing earlier and having the same AXI identifier is sent, so that the resource is wasted.

[0063] In the embodiment of the present application, a plurality of (for example, equal to the number of communication ports) specific execution units are selected from all execution units of the AI chip, each specific execution unit is associated with a respective communication port and is associated with a respective AXI identifier. Each specific execution unit sends data in a transaction of the AXI identifier associated with the specific execution unit based on the communication port associated with the specific execution unit, so as to avoid AXI identifier conflict in the AXI communication process. The remaining execution units in the AI chip other than the specific execution units are not associated with the communication port nor the AXI identifier, and do not have the capability of sending data in the AXI identifier transaction. Moreover, other types of tasks not belonging to the sending task are assigned to the remaining execution units, so as to ensure the sending efficiency of the specific execution units.

[0064] The above disclosure details the technical defects in the related art, the cause of the technical defects, and the thinking and analysis process of overcoming the technical defects. In fact, the cognition of the above technical defects is not the common knowledge in the art, but the novel discovery of the inventors in the research. In addition, the cause of the technical defects and the thinking and analysis process of overcoming the technical defects are also the gradual analysis results of the inventors in the actual research process, and are not the common knowledge in the art.

[0065] Figure 3 An exemplary flowchart of the task allocation method for the AI chip according to the embodiment of the present application. Figure 3The method can be performed by a processor in an AI chip. For example, the AI chip can be implemented as any one of a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), and a general-purpose graphics processing unit (GPGPU). For example, Figure 3 The method includes:

[0066] Step 101: determining a task to be executed by an AI chip, the AI chip including a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port.

[0067] The first execution unit can send data in transactions of the first AXI identifier associated with the first execution unit to outside the AI chip based on the first communication port associated with the first execution unit. The second execution unit is not associated with a communication port or an AXI identifier, and does not send data to outside the AI chip.

[0068] For example, the input task queue of the AI chip can be continuously monitored to determine the task to be executed by the AI chip. For example, the task to be executed can include:

[0069] (1) a computing task:

[0070] For example, the computing task can include a deep learning inference task, a deep learning training task, and a machine learning task.

[0071] The deep learning inference task can include: (a) a neural network forward propagation task: performing forward computation of a neural network on input data to obtain an output result; (b) a model deployment and inference task: deploying a trained deep learning model on a chip to perform inference computation on new data, such as performing inference on text data in natural language processing to generate a text summary or translation result.

[0072] The deep learning training task can include: (a) backpropagation and gradient calculation: calculating the gradient of the loss function with respect to the network weights for optimization and updating of the model, such as in training an image classification model, calculating the gradient to adjust the network parameters according to the difference between the predicted results and the true labels; (b) parameter updating and optimization: using the calculated gradient to update the network parameters through optimization algorithms (such as SGD, Adam, etc.) to improve the performance of the model.

[0073] The machine learning task can include: (a) feature extraction and conversion: feature extraction and conversion of raw data to improve the performance and efficiency of the model, such as in the data preprocessing stage, standardization, normalization, principal component analysis, etc. operations are performed on the data; (b) model training and prediction: training traditional machine learning models (such as SVM, decision tree, random forest, etc.) and using the trained model to predict and classify new data.

[0074] (2) Data processing task:

[0075] For example, the data processing task can include data preprocessing and data post-processing.

[0076] Data preprocessing can include: (a) data cleaning and denoising: removing noise and outliers in the data, filling or interpolating missing data to improve data quality; (b) data normalization and standardization: normalizing or standardizing the data to meet specific distribution or range requirements.

[0077] Data post-processing can include: (a) result arrangement and format conversion: arranging and converting the calculation or inference results to meet the requirements of subsequent processing or output, such as converting the output results of a deep learning model into readable text or image format; (b) data storage and management: storing and managing the processed data for subsequent queries and use, such as storing data in the chip's cache or external storage devices.

[0078] (3) Communication task:

[0079] For example, the communication task includes: (a) data sending task: sending the data generated inside the AI chip (such as calculation results, model parameters, etc.) to other chips or devices through the communication port (usually P2P communication port), such as in multi-GPU parallel computing, sending part of the calculation results to other GPUs for further processing; (b) data receiving task: receiving data from other chips or devices through the P2P communication port. Such as in distributed training, receiving gradient information or model parameter updates sent by other nodes.

[0080] The above exemplary describes the specific way of determining the task to be executed and the typical examples of the task to be executed, and those skilled in the art can realize that the description is only exemplary and does not limit the protection scope of the embodiments of the present application.

[0081] Step 102: determining a first communication task from the task to be executed, the first communication task indicating sending first data in the AI chip to outside the AI chip.

[0082] Here, the first communication task indicating sending first data in the AI chip to outside the AI chip is determined from the task to be executed. The first data can be any data in the AI chip. For example, the first data can include gradient information, updated model parameters or parameter update amount, intermediate calculation result or final settlement result, etc.

[0083] Step 103: assigning the first communication task to a first execution unit in the AI chip, wherein the first execution unit performs the first communication task based on the first communication port and the first AXI identifier to send the first data to outside the AI chip.

[0084] The AI chip can include a plurality of execution units having the capability of sending data to outside the AI chip. Each execution unit having the capability of sending data to outside the AI chip is associated with a respective AXI identifier and a respective communication port. Each execution unit having the capability of sending data to outside the AI chip also has the capability of receiving data from outside the AI chip.

[0085] For example, assigning the first communication task to the first execution unit includes selecting the first execution unit from all execution units having the capability of sending data to outside the AI chip according to the idle degree. For example, the execution unit with the largest idle degree is determined as the first execution unit. In an optional embodiment, the first execution unit is selected from all execution units having the capability of sending data to outside the AI chip based on communication delay. For example, the execution unit in the AI chip with the smallest communication delay with the first communication port is determined as the first execution unit. In an optional embodiment, the first execution unit is the execution unit in the AI chip with the best communication performance determined based on a communication performance monitoring tool or historical communication record.

[0086] By assigning the first communication task to the first execution unit associated with the first communication port and the first AXI identifier, the first execution unit can perform the first communication task based on the first communication port and the first AXI identifier to send the first data out of the AI chip. Specifically, the sending process of the first data can include: first, sending the write address and the write control information (for example, the first AXI identifier can be included in the write control information) out of the AI chip (for example, another AI chip) through the first communication port; and then, sending the first data carrying the first AXI identifier through the first communication port. Since the first communication port sending the first data is associated with the first AXI identifier, and other communication ports are not associated with the first AXI identifier, when the first communication port sends data of the first AXI identifier, there is no need to send data of the first AXI identifier in other communication ports, thereby avoiding identifier conflicts and overcoming or slowing down the problem of resource waste.

[0087] In one embodiment, the method further includes: determining a non-communication task (such as a calculation task or a data processing task, etc.) from the to-be-executed tasks; assigning the non-communication task to the second execution unit; and wherein the second execution unit executes the non-communication task. It can be seen that assigning the non-communication task to the second execution unit (rather than assigning it to the first execution unit) can improve the sending efficiency of the first execution unit. Here, the meaning of the non-communication task is: a task other than the communication task for transmitting data between AI chips. The communication task can include: sending data inside the AI chip to other chips through the P2P communication port, or receiving data from other chips through the P2P communication port. For example, the non-communication task can specifically include: a task that needs to read a High Bandwidth Memory (HBM) load (such as summing after reading data).

[0088] In one embodiment, assigning the non-communication task to the second execution unit includes: selecting the second execution unit based on the amount of available computing resources from all execution units that are not associated with any AXI identifier and any communication port. For example, the execution unit with the largest amount of available computing resources is determined as the second execution unit.

[0089] Specifically, when selecting the execution unit with the largest amount of available computing resources, the available computing resources can be measured based on any one or more of the following methods or any combination thereof.

[0090] Method (1): measuring the available computing resources based on a specific quantitative index.

[0091] For example, the execution unit with the largest number of available computing cores or the largest available memory capacity can be selected as the second execution unit.

[0092] Method (2): The available computing resources are measured based on the evaluation of the computing capability.

[0093] For example, the execution unit with the highest computing capability can be selected as the second execution unit.

[0094] Method (3): The available computing resources can be measured based on the estimation of the task execution time.

[0095] For example, the execution unit with the shortest task completion time when performing the same type of task can be estimated as the second execution unit.

[0096] Method (4): The available computing resources are dynamically evaluated based on the current load.

[0097] For example, the execution unit with the lowest load at the current moment can be selected as the second execution unit.

[0098] In one embodiment, the method further comprises: determining a second communication task from the tasks to be executed, the second communication task indicating receiving second data from outside the AI chip; and assigning the second communication task to the first execution unit; wherein the first execution unit performs the second communication task to receive the second data from outside the AI chip based on the first communication port.

[0099] Therefore, the first execution unit can not only perform the first communication task of sending the first data in the AI chip to outside the AI chip, but also perform the second communication task of receiving the second data from outside the AI chip. When performing the second communication task, the first execution unit directly receives the second data using the first communication port without associating the second data with the first AXI identifier. At this time, the data receiving task and the data sending task are respectively assigned to the same execution unit with communication capability.

[0100] In one embodiment, the AI chip comprises a third execution unit associated with a second AXI identifier and a second communication port of the AI chip, and the method comprises: determining a second communication task from the tasks to be executed, the second communication task indicating receiving second data from outside the AI chip; and assigning the second communication task to the third execution unit; wherein the third execution unit performs the second communication task to receive the second data from outside the AI chip based on the second communication port.

[0101] Therefore, the embodiments of the present application can also use the third execution unit different from the first execution unit to perform the second communication task of receiving the second data from outside the AI chip. When performing the second communication task, the third execution unit directly receives the second data using the second communication port without associating the second data with the second AXI identifier. At this time, the data receiving task and the data sending task are respectively assigned to different execution units with communication capability.

[0102] Figure 4 Figure 1 is a schematic diagram of a dual-card communication scenario according to an embodiment of the present application. Figure 4 In GPU0, SPC unit 0 (labeled as spc0), SPC unit 1 (labeled as spcl), SPC unit 2 (labeled as spc2) and SPC unit 3 (labeled as spc3) are associated with respective communication ports and respective AXI identifiers. In particular, spc0 is associated with P2P communication port 0 (labeled as P2P port 0) and AXI identifier 0, so that spc0 has the ability to send data in transactions based on P2P port 0 and AXI identifier 0 and also has the ability to receive data based on P2P port 0; spcl is associated with P2P communication port 1 (labeled as P2P port 1) and AXI identifier 1, so that spcl has the ability to send data in transactions based on P2P port 1 and AXI identifier 1 and also has the ability to receive data based on P2P port 1; spc2 is associated with P2P communication port 2 (labeled as P2P port 2) and AXI identifier 2, so that spc2 has the ability to send data in transactions based on P2P port 2 and AXI identifier 2 and also has the ability to receive data based on P2P port 2; spc3 is associated with P2P communication port 3 (labeled as P2P port 3) and AXI identifier 3, so that spc3 has the ability to send data in transactions based on P2P port 3 and AXI identifier 3 and also has the ability to receive data based on P2P port 3. The remaining execution units (spc4~spc7) in GPU0 are neither associated with P2P communication ports nor associated with AXI identifiers, and thus have no communication ability.

[0103] Since spc0~spc3 can all send data in transactions based on their associated communication ports and their associated AXI identifiers, AXI identifier conflicts in AXI communication can be avoided. Moreover, other types of tasks that are not sending tasks can be assigned to spc4~spc7, so that the sending efficiency of spc0~spc3 can be ensured.

[0104] The application also provides an AI chip. The AI chip comprises: a first execution unit associated with a first AXI identifier and a first communication port in the AI chip; a second execution unit not associated with any AXI identifier and any communication port; a scheduling unit configured to determine a task to be executed of the AI chip; determine a first communication task from the task to be executed, the first communication task indicating that first data in the AI chip is to be transmitted out of the AI chip; and assign the first communication task to the first execution unit; wherein the first execution unit is configured to execute the first communication task based on the first communication port and the first AXI identifier, so as to transmit the first data out of the AI chip.

[0105] The scheduling unit can be implemented as a combination of hardware and software. The core logic of the scheduling unit can be implemented by hardware circuit to ensure high speed and efficient task scheduling. At the same time, the scheduling unit can contain corresponding software or firmware support, so as to realize more complex scheduling algorithms and strategies, and work with other parts of the chip. For example, the scheduling unit can be embedded with a microprocessor or microcontroller to run scheduling algorithms and manage task queues.

[0106] Therefore, by assigning communication tasks and non-communication tasks to different execution units, the task processing efficiency of the chip is improved. At the same time, by associating specific communication tasks with specific AXI identifiers and communication ports, the efficiency and reliability of data transmission can be optimized.

[0107] In one embodiment, the scheduling unit is configured to determine a non-communication task from the task to be executed; and assign the non-communication task to the second execution unit; and the second execution unit is configured to execute the non-communication task.

[0108] The application is described below by taking all-reduce operation as an example. Figure 5 The following is a schematic diagram of performing all-reduce operation based on dual-card communication according to an embodiment of the application.

[0109] In Figure 5 , the high bandwidth memory (HBM) of GPU0 stores data a and data b, where data a is to-be-reduced data that needs to be transmitted to GPU1, and data b is to-be-reduced data that GPU0 itself needs. The HBM of GPU1 stores data c and data d, where data c is to-be-reduced data that needs to be transmitted to GPU0, and data d is to-be-reduced data that GPU1 itself needs.

[0110] The spc0 in GPU0 is associated with the communication port P2P port0 in GPU0, and the spc0 is associated with the AXI identifier 0. The spc1 in GPU1 is associated with the communication port P2P port1 in GPU1, and the spc1 is associated with the AXI identifier 5.

[0111] In GPU0, the communication task for sending data a to GPU1 is assigned to spc0. spc0 executes the communication task, specifically including: spc0 sends data a to P2P port 1 of GPU1 via communication port P2P port 0 with AXI identifier 0. spc1 of GPU1 writes data a into the workspace of HBM of GPU1.

[0112] In GPU1, the communication task for sending data c to GPU0 is assigned to spc1. spc1 executes the communication task, specifically including: spc1 sends data c to P2P port 0 of GPU0 via communication port P2P port 1 with AXI identifier 5. spc0 of GPU1 writes data c into the workspace of HBM of GPU0.

[0113] In GPU0, the execution unit with the largest amount of currently available computing resources is spc4. spc4 is assigned the rest of the tasks in the full reduction operation except the communication task, specifically including: (1) reading data b from HBM of GPU0 and reading data c from the workspace of GPU0; (2) calculating the reduction result of data b and data c (assuming b+c); (3) writing the reduction result into the workspace of HBM of GPU0.

[0114] In GPU1, the execution unit with the largest amount of currently available computing resources is spc5. spc5 is assigned the rest of the tasks in the full reduction operation except the communication task, specifically including: (1) reading data d from HBM of GPU1 and reading data a from the workspace of GPU1; (2) calculating the reduction result of data a and data d (assuming a+d); (3) writing the reduction result into the workspace of HBM of GPU1.

[0115] It can be seen that, by assigning the communication task to spc0 and spc1 which are specialized in performing communication, AXI identifier conflict is avoided. Moreover, by assigning the other tasks which are not communication tasks to spc4 and spc5 which do not need to perform communication, the communication efficiency of spc0 and spc1 is also ensured.

[0116] Figure 6 An exemplary structure diagram of a task assignment device of an AI chip according to an embodiment of the present application. As shown in FIG. 1, the AI chip includes a GPU0 and a GPU1. The GPU0 and the GPU1 are connected via a communication port P2P port 0 and a P2P port 1. The GPU0 and the GPU1 are connected via a communication port P2P port 2 and a P2P port 3. Figure 6As shown, the apparatus comprises: a first determining module configured to determine a task to be executed by an AI chip, the AI chip comprising a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port; a second determining module configured to determine a first communication task from the task to be executed, the first communication task indicating that first data in the AI chip is to be transmitted out of the AI chip; and an allocating module configured to allocate the first communication task to the first execution unit; wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier, so as to transmit the first data out of the AI chip.

[0117] In summary, in the embodiments of the present application, the task to be executed by the AI chip is determined, the AI chip comprising a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, and the second execution unit is not associated with any AXI identifier and any communication port; the first communication task is determined from the task to be executed, the first communication task indicating that first data in the AI chip is to be transmitted out of the AI chip; and the first communication task is allocated to the first execution unit; wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier, so as to transmit the first data out of the AI chip. Thus, the first execution unit transmits data in a transaction of an AXI identifier associated with the first execution unit based on a communication port associated with the first execution unit, avoids AXI identifier conflict in AXI communication, and saves resources.

[0118] In addition, other types of tasks that do not belong to the first communication task are allocated to other execution units, which can also ensure the transmission efficiency of the first execution unit.

[0119] The embodiments of the present application also provide an electronic device with a processor-memory architecture. Figure 7 is a structural diagram of an electronic device according to the embodiments of the present application. As shown in Figure 7As shown, the electronic device includes a processor 801, a memory 802, and a computer program stored on the memory 802 and executable on the processor 801, which, when executed by the processor 801, implements the task allocation method of the AI chip of any of the above. The memory 802 can be embodied as an electrically erasable programmable read-only memory (EEPROM), a flash memory, a programmable program read-only memory (PROM), or the like. The processor 801 can be embodied as one or more central processing units or one or more field programmable gate arrays, which integrate one or more central processing unit cores. Specifically, the central processing unit or the central processing unit core can be embodied as a CPU, a GPU, a GPGPU, an MCU, or a DSP, and the like.

[0120] The embodiments of the present application also provide a program product including a computer program, which, when executed by a processor, implements the task allocation method of the AI chip of any of the above. The program product can be stored in a computer readable storage medium. The computer readable storage medium includes, but is not limited to, an optical disc, a hard disk, a solid state disk, a USB flash disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk memory, an optical disk memory, and the like. These storage media can be connected to a computer system in a wired or wireless manner, so that the computer system can read the computer program product from the storage medium and load it into the memory for execution according to the needs.

[0121] It should be noted that not all steps and modules in the above processes and structural diagrams are necessary, and some steps or modules can be omitted according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The division of each module is only for the convenience of description of the adopted functional division, and in actual implementation, one module can be implemented by multiple modules, and the functions of multiple modules can be implemented by the same module. These modules can be located in the same device or in different devices.

[0122] The hardware modules in the embodiments can be implemented mechanically or electronically. For example, a hardware module can include a specifically designed permanent circuit or logic device (such as a dedicated processor, such as an FPGA or an ASIC) for completing a specific operation. For example, a specific operation can be completed in various types of chips (for example, an AI chip). A hardware module can also include a programmable logic device or circuit (such as a general-purpose processor or other programmable processor) temporarily configured by software for performing a specific operation. As to whether to implement a hardware module mechanically or by a dedicated permanent circuit or by a temporarily configured circuit (such as configured by software), it can be determined according to cost and time considerations.

[0123] The present application also provides a machine-readable storage medium storing instructions for causing a machine to perform the method described in the present application. Specifically, a system or apparatus equipped with a storage medium on which a software program code for realizing the functions of any of the above-described embodiments is stored, and a computer (or CPU or MPU) of the system or apparatus can be provided, and the computer (or CPU or MPU) is caused to read and execute the program code stored in the storage medium. In addition, part or all of the actual operations can be performed by an operating system or the like operating on the computer based on the instructions of the program code. The program code read out from the storage medium can also be written to a memory provided in a board inserted into the computer or a memory provided in an extension unit connected to the computer, and part or all of the actual operations can be performed by a CPU or the like mounted on the board or the extension unit based on the instructions of the program code, thereby realizing the functions of any of the above-described embodiments. The storage medium for providing the program code includes a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as a CD-ROM, a CD-R, a CD-RW, a DVD-ROM, a DVD-RAM, a DVD-RW, a DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer or a cloud over a communication network.

[0124] In the present specification, "schematically" means "to serve as an example, an instance, or an illustration", and any drawing, embodiment described as "schematic" in the present specification should not be interpreted as a more preferred or more advantageous technical solution. In order to make the drawing simple, only a part related to the present application is schematically shown in each drawing, and does not represent an actual structure as a product. In addition, in order to make the drawing simple and easy to understand, in some drawings, only one of components having the same structure or function is schematically shown, or only one of them is marked. In the present specification, "one" does not mean that the number of parts related to the present application is limited to "only one", and "one" does not mean that the number of parts related to the present application is excluded from the case where the number of parts related to the present application is "more than one". In the present specification, "upper", "lower", "front", "rear", "left", "right", "inner", "outer", and the like are used only to indicate relative positional relationships between the parts, and not to limit the absolute positions of the parts.

[0125] The above-described embodiments are merely preferred embodiments of the present application and are not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

1. A task allocation method for an AI chip, characterized in that, The method comprises: determining a task to be executed of an AI chip, the AI chip comprising a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, other communication ports are not associated with the first AXI identifier, and the second execution unit is not associated with any AXI identifier and any communication port; wherein the first execution unit is adapted to send data in a transaction of the associated first AXI identifier outside the AI chip based on the associated first communication port; the second execution unit does not send data outside the AI chip; wherein the AXI identifier is a unique identifier for distinguishing AXI transactions, and the AXI transaction is a complete data transmission operation on the AXI bus, and each AXI transaction carries an AXI identifier; determining a first communication task from the task to be executed, the first communication task indicating that first data in the AI chip is to be sent outside the AI chip; allocating the first communication task to the first execution unit; wherein the first execution unit executes the first communication task based on the first communication port and the first AXI identifier to send the first data outside the AI chip; wherein when the first communication port sends the first data, there is no need for other communication ports to send data of the first AXI identifier; Further comprising: determining a non-communication task from the task to be executed; allocating the non-communication task to the second execution unit; wherein the second execution unit executes the non-communication task.

2. The method of claim 1, wherein, The allocation of the non-communication task to the second execution unit comprises: selecting an execution unit with the largest amount of available computing resources from all execution units not associated with any AXI identifier and any communication port; determining the execution unit with the largest amount of available computing resources as the second execution unit.

3. The method of claim 1, wherein, Further comprising: determining a second communication task from the task to be executed, the second communication task indicating that second data is to be received from outside the AI chip; allocating the second communication task to the first execution unit; wherein the first execution unit executes the second communication task based on the first communication port to receive the second data from outside the AI chip.

4. The method of claim 1, wherein, The AI chip comprises a third execution unit associated with a second AXI identifier and a second communication port in the AI chip, and the method comprises: determining a second communication task from the task to be executed, the second communication task indicating that second data is to be received from outside the AI chip; allocating the second communication task to the third execution unit; wherein the third execution unit executes the second communication task based on the second communication port to receive the second data from outside the AI chip.

5. The method according to any one of claims 1-4, wherein: the first execution unit is an execution unit in the AI chip with the smallest communication delay for the first communication port; or the first execution unit is an execution unit in the AI chip with the smallest communication delay for the first communication port; or The first execution unit is an execution unit with the best communication performance in the AI chip determined based on a communication performance monitoring tool or historical communication records. 6.A task allocation apparatus of an AI chip, characterized by comprising: The device comprises: A first determination module is configured to determine a to-be-executed task of an AI chip, the AI chip comprising a first execution unit and a second execution unit, wherein the first execution unit is associated with a first AXI identifier and a first communication port in the AI chip, other communication ports are not associated with the first AXI identifier, and the second execution unit is not associated with any AXI identifier and any communication port; the first execution unit is adapted to send data in a transaction of the associated first AXI identifier to outside the AI chip based on the associated first communication port; the second execution unit does not send data to outside the AI chip; the AXI identifier is a unique identifier for distinguishing AXI transactions, and the AXI transaction is a complete data transmission operation performed on an AXI bus, each AXI transaction carrying an AXI identifier; A second determination module is configured to determine a first communication task from the to-be-executed task, the first communication task indicating that first data in the AI chip is to be sent to outside the AI chip; An allocation module is configured to allocate the first communication task to the first execution unit; the first execution unit performs the first communication task based on the first communication port and the first AXI identifier to send the first data to outside the AI chip; when the first communication port sends the first data, there is no need for other communication ports to send data of the first AXI identifier; a non-communication task is determined from the to-be-executed task; the non-communication task is allocated to the second execution unit; the second execution unit performs the non-communication task.

7. An AI chip, comprising: Comprise: A first execution unit is associated with a first AXI identifier and a first communication port in the AI chip; other communication ports are not associated with the first AXI identifier; the first execution unit is adapted to send data in a transaction of the associated first AXI identifier to outside the AI chip based on the associated first communication port; when the first communication port sends first data, there is no need for other communication ports to send data of the first AXI identifier; the AXI identifier is a unique identifier for distinguishing AXI transactions, and the AXI transaction is a complete data transmission operation performed on an AXI bus, each AXI transaction carrying an AXI identifier; A second execution unit is not associated with any AXI identifier and any communication port; the second execution unit does not send data to outside the AI chip; A scheduling unit is configured to determine a to-be-executed task of an AI chip; determine a first communication task from the to-be-executed task, the first communication task indicating that first data in the AI chip is to be sent to outside the AI chip; allocate the first communication task to the first execution unit; The first execution unit is configured to execute the first communication task based on the first communication port and the first AXI identifier, and send the first data to outside of the AI chip. The scheduling unit is configured to determine a non-communication task from the to-be-executed tasks, and assign the non-communication task to the second execution unit. The second execution unit is configured to execute the non-communication task.

8. An electronic device, comprising: Comprise: a memory; a processor; The memory stores an application executable by the processor, which is configured to enable the processor to execute the task allocation method of the AI chip according to any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer readable instructions, which, when executed by a processor, enable the processor to execute the task allocation method of the AI chip according to any one of claims 1-5.

10. A program product comprising a computer program, characterized in that The computer program, when executed by a processor, implements the task allocation method of the AI chip according to any one of claims 1-5.

Images