Task processing method, system and storage medium

Abstract

The present application relates to a task processing method, which is applied to a system including N devices, wherein N is greater than or equal to 2; the method comprises: when a preset condition is met, performing calculations according to a data block stored in the preset storage space to obtain a calculation result.

Description

Technical Field

[0001] The present application relates to the field of chip technology, and in particular to a task processing method, system and storage medium. Background Art

[0002] With the development of high-performance computers and data centers, more and more computing resources are being centralized, and multi-device collaborative processing has become the norm. When multiple devices are running in coordination, they often need to interact with each other, such as data transmission interaction, etc. How to achieve the execution efficiency of the coordinated operation of multiple devices deserves attention. Summary of the invention

[0003] Based on this, it is necessary to provide a task processing method, system and storage medium to address the above technical problems, which can achieve efficient processing of small data between multiple devices, reduce processing delays and improve processing efficiency.

[0004] A task processing method is applied to a system including N devices, wherein N is greater than or equal to 2; the method comprises:

[0005] Receiving a data block transmitted by an upstream device, and storing the data block in a preset storage space;

[0006] Transmitting the data block received from the upstream device to the downstream device of the current device;

[0007] The above operations are repeatedly performed until a preset condition is met, and when the preset condition is met, calculations are performed according to the data blocks stored in the preset storage space to obtain calculation results.

[0008] In one embodiment, the method further comprises:

[0009] When the above operations are repeatedly performed for a preset number of times, it is determined that the preset condition is satisfied.

[0010] In one of the embodiments, before receiving the data block transmitted by the upstream device, the method further includes:

[0011] Sending a data block to a downstream device of the current device according to a received data transmission trigger signal;

[0012] The data transmission trigger signal is sent by a driver to the N devices.

[0013] In one embodiment, the data block carries communication data; and transmitting the data block received from the upstream device to the downstream device of the current device includes:

[0014] The data block received from the upstream device is transmitted to the downstream device according to the communication data in the received data block transmitted by the upstream device.

[0015] In one embodiment, the communication data includes a first communication descriptor; and transmitting the data block received from the upstream device to the downstream device of the current device includes:

[0016] The data block received from the upstream device is transmitted to the downstream device according to the first communication descriptor in the received data block transmitted by the upstream device.

[0017] In one embodiment, transmitting the data block received from the upstream device to the downstream device according to the first communication descriptor in the received data block transmitted by the upstream device includes:

[0018] Obtaining a first task template according to a first communication descriptor received and transmitted by the upstream device;

[0019] According to the first task template, obtaining task forwarding information of the current device;

[0020] The data block received from the upstream device is transmitted to the downstream device according to the task forwarding information.

[0021] In one of the embodiments, the communication data further includes a second communication descriptor; and the method further includes:

[0022] According to the second communication descriptor in the received data block transmitted by the upstream device, stop sending the data block to the downstream device of the current device.

[0023] In one embodiment, the method further comprises:

[0024] Obtaining a second task template according to the received second communication descriptor transmitted by the upstream device;

[0025] According to the second task template, obtaining task execution information of the current device;

[0026] An operation is performed according to the task execution information of the current device and the data block stored in the preset storage space to obtain an operation result.

[0027] In one of the embodiments, the first task template and the second task template are configured to each of the devices by a driver.

[0028] In one embodiment, the current device includes a task scheduling device and an artificial intelligence processing device; the current device performs calculations according to the task processing information of the current device and the data blocks stored in the preset storage space to obtain calculation results, including:

[0029] The task scheduling device of the current device triggers the artificial intelligence processing device of the current device to start according to the task execution information of the current device;

[0030] The artificial intelligence processing device of the current device performs calculations according to the data blocks stored in the preset storage space to obtain calculation results.

[0031] In one embodiment, the method further comprises:

[0032] The calculation result is stored from the preset storage space to the cache space.

[0033] In one of the embodiments, the first communication descriptor and the second communication descriptor are configured by a driver.

[0034] In one of the embodiments, the preset storage space of the current device is allocated by a driver.

[0035] In one of the embodiments, the size of the data block is less than or equal to a preset threshold.

[0036] The present application also provides a system, comprising N devices connected to each other, wherein N is greater than or equal to 2;

[0037] The N devices respectively send data blocks to downstream devices connected thereto;

[0038] The N devices store the data blocks received from the upstream device in preset storage spaces respectively;

[0039] Repeat the above operations until the preset conditions are met;

[0040] When the preset conditions are met, the N devices respectively perform operations according to the data blocks stored in the preset storage space to obtain operation results.

[0041] In one embodiment, each of the devices includes a communication module, and the communication module includes:

[0042] A receiving device, configured to receive a data block transmitted by an upstream device; and

[0043] The sending device is used to transmit the data block received from the upstream device to the downstream device of the device.

[0044] In one embodiment, each data block includes communication data; each of the devices further includes:

[0045] The task scheduling device is used to determine task processing information according to the communication data transmitted by the receiving device.

[0046] In one of the embodiments, the communication data includes a first communication descriptor;

[0047] The task scheduling device is also used to obtain a first task template according to the received first communication descriptor, and determine task forwarding information according to the first task template to trigger the sending device to transmit the data block received from the upstream device to the downstream device of each of the devices.

[0048] In one of the embodiments, the communication data includes a second communication descriptor; each of the devices also includes an artificial intelligence processing device;

[0049] The task scheduling device is also used to obtain a second task template based on the received second communication descriptor, and determine task processing information based on the second task template to trigger the artificial intelligence processing device to perform calculations based on the task processing information of the current device and the data blocks stored in the preset storage space to obtain calculation results.

[0050] The present application also provides a computer storage medium, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it can implement any of the methods described above.

[0051] The task processing method, system and storage medium of the present application first transmit data between N devices, and save the received data blocks in the local preset storage space of each device, and when the preset conditions are met, each device performs operations according to the data blocks stored in the preset storage space, thereby reducing the frequent startup of the computing components in each device by separating data transmission and operation, thereby achieving the delay of task processing and improving the task processing efficiency of the multi-device system. At the same time, the task processing method, system and storage medium can also fully utilize the computing resources in each device to avoid idleness or waste of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A structural diagram of a system for interconnecting multiple devices according to an embodiment of the present application;

[0053] Figure 2 A structural diagram of a system for interconnecting multiple devices according to an embodiment of the present application;

[0054] Figure 3A schematic diagram of data transmission between devices in a system of interconnecting multiple devices according to an embodiment of the present application;

[0055] Figure 4 A storage hierarchy diagram of each device in a system in which multiple devices are interconnected in one embodiment of the present application;

[0056] Figure 5 A flowchart of a task processing method according to an embodiment of the present application;

[0057] Figure 6 A flowchart of a task processing method according to an embodiment of the present application;

[0058] Figure 7 A flowchart of data forwarding in a task processing method according to an embodiment of the present application;

[0059] Figure 8 The flowchart is a task execution method in a task processing method according to an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] It should be understood that the terms "first", "second", etc. in the claims, specifications and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0062] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0063] like Figure 1 and Figure 2 As shown, an embodiment of the present application provides a system for interconnecting multiple devices. Figure 1 As shown in (A), multiple devices are connected end to end to form a ring structure (Ring interconnection structure). Figure 1As shown in (B), each device can be connected to other devices in the system. It should be clear that the "connection" between devices here can be a communication connection. Figure 1 The following mainly combines the two device interconnection systems of FIG. Figure 1 The system shown in (A) exemplifies the working principle of a system with multiple devices interconnected.

[0064] Among them, the multi-device interconnected system may include N devices, N is greater than or equal to 2. Devices may include chips, boards, processors and processor cores, and devices for communication between devices include but are not limited to data exchange between chips, boards, processors or processor cores. Optionally, each device may include an artificial intelligence processing device, wherein the artificial intelligence processing device includes but is not limited to acceleration devices such as graphics processing units GPU (Graphics Processing Unit), neural network processors NPU (neural-network processing units). The device may also include other structures such as a central processing unit CPU and storage, which are not specifically limited here.

[0065] like Figure 2 As shown, the N devices in the system can be connected to a host, which can be a general-purpose processor. Each device has a communication task processing system 200. Optionally, Figure 2 The devices in (device 1, device 2, device 3, and device N) can communicate in two ways:

[0066] Mode 1: Realize inter-device communication through the host (normal transmission mode). When data transmission between device 1 and device 2 is realized through mode 1, the host first controls the sending device 220 of device 1 to send relevant data to device 2, and the receiving device 240 of device 2 returns status information to the host after receiving the relevant data. That is, in the case of mode 1, data transmission between different devices requires frequent intervention of the host.

[0067] Method 2: Realize inter-device communication through the communication task processing system on each device (hardware accelerated sending mode). Specifically, when realizing data transmission between devices through method 2, it is only necessary to intervene through the host in the initial stage, and then different devices can directly communicate data without the intervention of the host, thereby speeding up the data transmission efficiency. For example, when realizing data transmission between device 1 and device 2 through method 2, first, the host controls the sending device 220 of device 1 to send relevant data to device 2, and after receiving the relevant data, the receiving device 240 of device 2 can directly trigger the sending device 220 of device 2 to transmit the relevant data to device 3. The embodiments of the present application are mainly described in conjunction with the above-mentioned hardware acceleration mode.

[0068] like Figure 2 As shown, the task processing system 200 on each device may include a memory 210, a task processing device 230 and a communication module. The memory 210 may be used to store data blocks involved in the task, device status data of each device, communication data and communication task templates, etc. The communication module is used for data communication between devices, and the communication module may include a sending device 220 and a receiving device 240. The sending device 220 of each device is used to send data to the downstream device of the device, and the receiving device 240 of each device is used to receive data sent by the upstream device of the device. The task processing device of each device is used to process the data received by the receiving device of the device.

[0069] Furthermore, the task processing device 230 of each device may include a task scheduling device and an artificial intelligence processing device. The task scheduling device may determine the task processing information based on the data block received from the sending device 220, and trigger the sending device 220 and / or the artificial intelligence processing device on the device based on the task processing information. The data block may include communication data, and the task scheduling device may determine the task processing information based on the communication data, and the task processing information may include task forwarding information and task execution information.

[0070] Specifically, the task scheduling device of each device can trigger the action of the sending device 220 of the current device according to the task forwarding information, so that the sending device 220 of the current device transmits data to its downstream device, thereby realizing the broadcast or distribution operation of data between devices. The task scheduling device of each device can also trigger the action of the artificial intelligence processing device according to the task execution information, and the artificial intelligence processing device can be used to perform related operations according to the task execution information. Among them, the related operations may include merge operations (Reduce), all-reduce operations (Allreduce) and scattered merge operations (Reduce-scatter), etc., which are not specifically limited here. Furthermore, the task scheduling device of each device can also trigger the action of the sending device 220 of the device according to the task execution information after the artificial intelligence processing device completes the related operations, so as to send the operation results of the artificial intelligence processing device to other devices.

[0071] Optionally, before data is transmitted between devices, the host may configure corresponding data blocks for each device, and the size of the data blocks on each device may be the same or different, which is not specifically limited here. Specifically, the driver running on the host may first apply for a preset storage space on each device to store data blocks during data transmission, and respectively move the data blocks corresponding to each device from the host memory to the preset storage space of the device memory (memory 210) on each device, thereby realizing the allocation of data blocks on each device. Further optionally, the size of the preset storage space on each device may be greater than or equal to the size of N data blocks of N devices, that is, the preset storage space can accommodate N data blocks. In other embodiments, the size of the preset storage space may also be determined according to actual conditions, which is not specifically limited here.

[0072] Optionally, data transmission between multiple devices in the system may be performed simultaneously, and the current device may refer to each device in the system. Figure 2 As shown, the sending device 220 of device 1 can send the data block thereon to device 2, and the sending device 220 of device 2 can send the data block thereon to device 3 at the same time, ... The sending device 220 of device N-1 can send the data block thereon to device N at the same time, and the sending device 220 of device N can send the data block thereon to device 1 at the same time. After receiving the data block transmitted by the upstream device, each device can automatically forward the data block to its downstream device according to the communication data carried in the data block. This cycle of n rounds of data transmission is repeated until the preset condition is met. When the preset condition is met, the system can stop the data transmission between devices. Optionally, n can be equal to N-1. In other possible embodiments, n can also take other possible values, which are not specifically limited here. Optionally, after N-1 times of data transmission, it can be determined that the preset condition is met.

[0073] Furthermore, when the preset conditions are met, each device can perform operations according to the N data blocks stored in its preset storage space to obtain operation results. Specifically, when the preset conditions are met, the task scheduling device on each device can trigger the artificial intelligence processing device thereon to start, and the artificial intelligence processing device can perform operations according to the N data blocks stored in its preset storage space to obtain operation results. In the embodiment of the present application, the data volume of each data block can be less than or equal to a preset threshold, and the preset threshold can be determined based on the time it takes for the data block to be transmitted from one device to another. For example, when the time it takes for each data block to be transmitted from one device to another is less than the time it takes for each device to perform operations based on the data it receives, the data volume of the task data block can be less than or equal to the preset threshold.

[0074] The computing process often needs to wait for the data transmission to be completed before it can proceed, which easily causes the computing resources on the device to be idle, thereby extending the execution time of the entire task. In the embodiment of the present application, according to the characteristics of the data involved in the task, before the execution of the operation, a certain amount of data transmission is completed in a centralized manner, and then a certain number of data blocks are operated as a whole, thereby shortening the delay during the execution of the task, thereby improving the execution efficiency of the task. At the same time, the task processing method in the embodiment of the present application can also reduce the startup overhead of computing components such as artificial intelligence processors in the device, further improving the execution efficiency of the task. Optionally, the time consumed by the certain amount of data transmission is substantially equal to the time consumed by the operation. For example, the time consumed by the certain amount of data transmission can be less than or equal to the time consumed by the operation.

[0075] like Figure 3 As shown in the figure, before multiple devices perform data transmission, the data status of each device is as follows: Figure 3 As shown in (A), after the first data transmission, the data status of each device is as follows: Figure 3 As shown in (B), after the second data transmission, the data status of each device is as follows: Figure 3 As shown in (C), after the N-1th data transmission, the data status of each device is as follows: Figure 3 (D) is shown. Afterwards, each device can perform operations based on the data blocks stored in its preset storage space to obtain operation results.

[0076] Optionally, before each device performs a calculation, each device may also determine whether the data required for the calculation is complete. When it is determined that the data required for the calculation is complete, each device may perform the calculation operation and obtain the calculation result. Optionally, each device may send an interrupt signal to the host, and the host may return a feedback signal to the current device indicating whether the data is complete, and the current device may determine whether to perform the calculation operation based on the feedback signal. Further, optionally, after each device completes the calculation operation and obtains the calculation result, it may also send a calculation completion signal to the host device.

[0077] Optionally, each device may also store the calculation results in a cache space. In the embodiment of the present application, by storing the calculation results in the cache space, without directly storing the calculation results in the external memory, the IO memory access overhead is reduced. In this way, when the calculation results need to be reused, the calculation results can be directly read from the cache space, thereby improving the calculation efficiency.

[0078] like Figure 4 As shown, the memory on each device can be divided into three levels: a first storage space, a cache space, and a second storage space, wherein the size of the storage space of the first storage can be smaller than the size of the cache space, and the size of the cache space can be smaller than the storage space of the second storage. The first storage space can be an on-chip storage space, the cache space can be LLC (Last Level Cache), and the second storage space can be DDR (Double Data Rate). The above-mentioned preset storage space can be a space on the first storage. After completing the operation, each device can temporarily store the operation results in the cache space without storing the operation results in the second storage space, thereby reducing the overhead of IO memory access.

[0079] Optionally, since the storage space of the cache space is relatively small, when the cache space is used to store calculation results, the cache space can be locked by software or hardware. Once the cache space is locked, other data accesses cannot occupy the cache space.

[0080] Furthermore, the above operation results can be used as input data required for the next round of operations. In one embodiment, the driver can send the operation results stored in the cache space of each device as the initial data block to each device. At this time, each device can directly obtain the corresponding data block from the cache space without having to obtain the corresponding data block from the second storage space or the host memory, thereby improving the data access efficiency and further improving the efficiency of task processing. In another embodiment, the above operation results can be directly forwarded to the downstream device of the device through the sending device on the device.

[0081] Optionally, when the size of a data block is greater than a preset threshold, when each device receives a data block transmitted by an upstream device, the task scheduling device on each device may trigger the start of its artificial intelligence processing device, and the artificial intelligence device may perform the above-mentioned related operations based on the data block it receives, and store the operation results in the second storage space. In the embodiment of the present application, since the data block is relatively large, the time it takes to transmit the data block from one device to another is equivalent to the time it takes for the artificial intelligence processing device to perform the operation. At this time, each device can start the artificial intelligence device to perform the operation after receiving the data block, which can reduce the processing delay on the one hand, and avoid storing N data blocks on the device, thereby reducing the storage space occupied by the device.

[0082] Optionally, to further improve processing efficiency, when the artificial intelligence processing device of each device is performing calculations, the sending device or receiving device on each device can simultaneously transmit data. When the artificial intelligence processing device of each device is performing calculations, the data block can also be moved at the same time. Through the parallel operation of different devices on the device, a pipeline operation mode of various processing operations can be realized, thereby reducing processing delays and improving processing efficiency.

[0083] Optionally, the communication data in each data block may be a communication descriptor, which is used to record information related to the data block. Optionally, the communication descriptor may include: task type (task_class), task identifier, communication task template address, communication task template identifier, source address and destination address of data, etc. The communication descriptor may be configured to each device by a driver running on the host before multiple devices interact with each other. The communication descriptor may be bound to other data to form a data block in the data transmission process. Optionally, the data block on each device may be split into at least one sub-data block, and each device may transmit the data according to the size of a sub-data block. Each sub-data block includes the above-mentioned communication descriptor, and each sub-data block may be called a communication slice.

[0084] At this time, before data transmission between devices, the driver can apply for a preset storage space on each device that can accommodate N communication slices, and store a communication slice in the preset storage space of each device respectively. Afterwards, the host can send a data transmission start signal to each device, and each device starts sending communication slices to its downstream device according to the data transmission signal it receives. After completing one data transmission, each device can automatically trigger the next data transmission according to the communication slice it receives. Specifically, each device can automatically trigger the next data transmission according to the communication descriptor in the communication slice it receives, until the data transmission between devices is stopped when the preset conditions are met. Each device performs operations based on the data blocks contained in the communication slice in its preset storage space.

[0085] Specifically, the first data transmission between multiple devices is triggered by the driver, and the driver can send a data transmission trigger signal to the receiving device 240 of each device. After each device receives the data transmission trigger signal sent by the driver, the sending device 220 of each device starts and sends a communication slice to its downstream device. The receiving device 240 of each device can receive the communication slice transmitted by the sending device 220 of its upstream device, and save the communication slice transmitted by the upstream device to the preset storage space.

[0086] Furthermore, the other n-1 data transmissions after the first transmission can be automatically triggered by the task scheduling device on the device according to the communication data (such as the communication descriptor) in the data block. The receiving device 240 of each device can also transmit the communication descriptor in the communication slice to the task scheduling device of the device, and the task scheduling device of the device can trigger its sending device 220 to send the received communication slice to the downstream device according to the communication descriptor. In the embodiment of the present application, other data transmissions other than the first transmission between multiple devices can be directly triggered by the task scheduling device, thereby reducing the interaction between the device and the host and improving the task processing efficiency.

[0087] Specifically, the task scheduling device can determine the task processing information based on the communication descriptor it receives, and the task processing information may include task forwarding information and task execution information. The above-mentioned communication descriptor may include a first communication descriptor and a second communication descriptor, wherein the first communication descriptor is used to represent the communication descriptor corresponding to the first n-1 data transmissions, and the second communication descriptor is used to represent the communication descriptor corresponding to the nth data transmission. n represents the total number of data transmissions performed before each device performs data operations, n is greater than or equal to 2, and n is a positive integer. Optionally, n is equal to N-1.

[0088] The task scheduling device can determine that the task processing information is task forwarding information based on the first communication descriptor it receives, so that the task scheduling device can automatically trigger the device's sending device 220 to send the received communication slice to the downstream device. The task scheduling device can also determine that the task processing information is task execution information based on the second communication descriptor it receives, so that the task scheduling device can automatically trigger the device's artificial intelligence processing device to start, so that the artificial intelligence processing device starts to perform operations.

[0089] Further, the task scheduling device can obtain a task template according to the communication descriptor it receives, and determine the task processing information according to the task template, and the task processing information may include task forwarding information and task execution information. Accordingly, the task template may also include a first task template and a second task template, wherein the first task template is used to configure the control information of forwarding the data received by the receiving device 240 of the device to the downstream device, and the first task template may include a forwarding flag and a forwarding path configured by software (the forwarding path may include the identifier of the downstream device, etc.). Optionally, there may be more than one downstream device of each device, and the device may determine which downstream devices each device needs to send data to through the downstream device identifier in its first task template. The task scheduling device can obtain the first task template according to the first communication descriptor it receives, and determine the task forwarding information according to the first task template, so that the task scheduling device can automatically trigger the sending device 220 of the device to send the communication slice it receives to the downstream device. Among them, the first communication descriptor may include the template identifier of the first task template, and the task scheduling device can obtain the first task template according to the template identifier of the first task template. Optionally, the task scheduling device may also determine whether to execute the forwarding task and the forwarding path of the forwarding task according to the forwarding flag in the first task template, and obtain the task forwarding information.

[0090] The second task template is used to configure the configuration information for the artificial intelligence processing device of the device to perform operations. The task scheduling device can also obtain the second task template based on the second communication descriptor it receives, and determine the task execution information based on the second task template, so that the task scheduling device can automatically trigger the artificial intelligence processing device of the device to start, so that the artificial intelligence processing device starts to perform operations. Among them, the second communication descriptor contains the template identifier of the second task template, and the task scheduling device can obtain the second task template based on the template identifier of the second task template.

[0091] Optionally, the task template (including the first task template and the second task template) can be configured by the driver to each device before data is transmitted between the devices. Furthermore, the task template can be configured to the task scheduling device of each device. Specifically, the driver can send the task template to each device respectively, each device can return the template identifier of each task template to the driver, and the driver can write the template identifier of each task template into the corresponding communication descriptor.

[0092] Optionally, the communication descriptor can be configured by the driver before data is transmitted between devices. Specifically, the driver can send Rx tasks to each device respectively, and after each device receives the Rx task, the downstream device can transmit its corresponding task template identifier and the address of the preset storage space corresponding to the downstream device to its upstream device. Afterwards, the driver can also send Tx tasks to each device respectively, and configure the communication descriptor to the current device according to the task template identifier transmitted by the downstream device of the current device and the address of the preset storage space corresponding to the downstream device.

[0093] like Figure 5 As shown, an embodiment of the present application further provides a task processing method, which is applied to the above-mentioned system including N devices, wherein N is greater than or equal to 2; the method includes:

[0094] S510: Receive a data block transmitted by an upstream device, and store the data block in a preset storage space.

[0095] Optionally, before data transmission between devices, the driver can apply for a preset storage space on the memory of the current device, and the preset storage space can be used to store data blocks sent by the host to the current device, as well as data blocks transmitted by other devices during data transmission between devices. Optionally, the space size of the preset storage space can store a preset number of data blocks, and the preset number can be equal to the number of devices N, that is, the space size of the preset storage space can store N data blocks on N devices.

[0096] Optionally, before data transmission between devices, the driver can also send a data block to the current device. Specifically, the driver can move the data block stored in the host memory to the preset storage space of the current device. In this way, during the data transmission between devices, the current device can transmit the data block stored in its preset storage space to its downstream device.

[0097] The current device refers to a device that is performing data transmission. Optionally, N devices in the system can perform data transmission simultaneously, and each device in the system can serve as the current device. The current device can receive data blocks transmitted by its upstream device and store the data blocks received from the upstream device in a preset storage space of the current device.

[0098] Optionally, the size of the data block may be less than or equal to a preset threshold. The preset threshold may be determined based on the time taken for the data block to be transmitted from one device to another. For example, when the time taken for the data block to be transmitted from one device to another is less than or equal to the time taken for the operation, it may be considered that the data size of the data block is less than or equal to the preset threshold. In other embodiments, the preset threshold may also be a preset fixed value, such as 1KB, etc., which is not specifically limited here.

[0099] S520: Transmit the data block received from the upstream device to the downstream device of the current device.

[0100] Optionally, the data block may include communication data, which can be used to trigger data transmission between devices. After receiving the data block transmitted by its upstream device, the current device can automatically transmit the data block received from the upstream device to the downstream device of the current device according to the communication data carried in the data block. In an embodiment of the present application, after receiving a data block, the current device can automatically trigger a data forwarding task according to the communication data carried in the data block without the intervention of the host, thereby reducing the frequent interaction between the host and the current device, thereby improving the efficiency of task execution.

[0101] S540: The current device determines whether a preset condition is met.

[0102] Optionally, after operations S520 and S530 are repeatedly performed for a preset number of times, it is determined that the preset condition is satisfied. The preset number n may be equal to N-1, and at this time, N data blocks may be stored in the preset storage space of each device. In other embodiments, the preset condition may also be determined according to actual conditions, which is not specifically limited here.

[0103] When the preset condition is not met, operations S520 and S530 are repeatedly performed until the preset condition is met. When the preset condition is met, operation S550 is performed to perform calculations according to the data blocks stored in the preset storage space to obtain calculation results.

[0104] Since the current device can automatically forward the data block to its device when it receives a data block, Figure 1In the ring system structure shown, the current device can repeatedly perform the above operations S520 and S530 until the preset condition is met. When the preset condition is met, the current device can perform operations according to the data blocks stored in the preset storage space to obtain operation results.

[0105] Furthermore, when the preset conditions are met, the task scheduling device of the current device can trigger the artificial intelligence processing device on the current device to start, and the artificial intelligence processing device of the current device can perform operations according to the data blocks stored in the preset storage space to obtain operation results. The operations may include reduce operations, all-reduce operations, and reduce-scatter operations, etc., which are not specifically limited here.

[0106] Since the operation process often needs to wait for the data transmission to be completed before it can be carried out, it is easy to cause the idleness of the computing resources on the device, thereby extending the execution time of the entire task. In the embodiment of the present application, according to the characteristics of the data involved in the task, before the operation is executed, a certain amount of data transmission is completed in a centralized manner, and then a certain number of data blocks are operated as a whole, thereby shortening the delay during the task execution process, thereby improving the execution efficiency of the task. At the same time, the task processing method in the embodiment of the present application can also reduce the startup overhead of computing components such as artificial intelligence processors in the device, further improving the execution efficiency of the task. Optionally, the time consumed by the certain amount of data transmission is substantially equal to the time consumed by the operation. For example, the time consumed by the certain amount of data transmission can be less than or equal to the time consumed by the operation.

[0107] like Figure 3 As shown in the figure, before multiple devices perform data transmission, the data status of each device is as follows: Figure 3 As shown in (A), after the first data transmission, the data status of each device is as follows: Figure 3 As shown in (B), after the second data transmission, the data status of each device is as follows: Figure 3 As shown in (C), after the N-1th data transmission, the data status of each device is as follows: Figure 3 (D) is shown. Afterwards, each device can perform operations based on the data blocks stored in its preset storage space to obtain operation results, such as Figure 3 (E) as shown.

[0108] Alternatively, if Figure 6 As shown, the method also includes:

[0109] S560, the current device stores the operation result from the preset storage space to the cache space. In the embodiment of the present application, by storing the operation result in the cache space, there is no need to store the operation result directly in the external memory, thereby reducing the IO memory access overhead. In this way, when the operation result needs to be reused, the operation result can be directly read from the cache space, thereby improving the operation efficiency.

[0110] like Figure 4 As shown, the memory on each device can be divided into three levels: a first storage space, a cache space, and a second storage space, wherein the size of the storage space of the first storage can be smaller than the size of the cache space, the size of the cache space can be smaller than the storage space of the second storage, and the second storage space can be DDR (Double Data Rate). The above-mentioned preset storage space can be the space on the first storage. After completing the operation, each device can temporarily store the operation results thereon in the cache space without storing the operation results in the second storage space, thereby reducing the IO memory access overhead. Optionally, since the storage space of the cache space is relatively small, when using the cache space to store the operation results, the cache space can be locked by software or hardware. Once the cache space is locked, other data accesses cannot occupy the cache space.

[0111] Furthermore, the above operation results can be used as input data required for the next round of operation. In one embodiment, the driver can send the operation results stored in the cache space of each device as the initial data block to each device. At this time, each device can directly obtain the corresponding data block from the cache space without having to obtain the corresponding data block from the second storage space or the host memory, thereby improving the data access efficiency and further improving the efficiency of task processing. In another feasible embodiment, the operation result can also be forwarded to its downstream device through the sending device of the current device.

[0112] Optionally, the first data transmission between devices may be triggered by the host, and thereafter, n-1 data transmissions between devices may be automatically triggered by the task scheduling device in the device, thereby reducing the interaction between the host and the device. Figure 5 and Figure 6 As shown, before the current device receives the data block transmitted by the upstream device, the above method may further include:

[0113] S510. The current device sends a data block to a downstream device of the current device according to a received data transmission trigger signal; wherein the data transmission trigger signal is sent to the N devices by a driver.

[0114] Optionally, the communication data in each data block may be a communication descriptor, which is used to record information related to the data block. Optionally, the communication descriptor may include: task type (task_class), task identifier, communication task template address, communication task template identifier, source address and destination address of data, etc. The communication descriptor may be configured by a driver running on a host to each device before multiple devices interact with each other, and the communication descriptor may be bound with other data to form a data block in the data transmission process. Optionally, the data block on each device may be split into at least one sub-data block, and each device transmits data according to the size of a sub-data block. Each sub-data block includes the above-mentioned communication descriptor, and each sub-data block may be called a communication slice. When data is interacted between devices, the devices transmit according to the communication slice, and each communication slice may include a communication descriptor.

[0115] Optionally, the current device may forward the data block to a downstream device of the current device according to the received data block. The operation of the current device automatically forwarding the data block to the downstream device of the current device may include:

[0116] The current device transmits the data block received from the upstream device to the downstream device according to the communication data in the data block transmitted by the upstream device. Specifically, the current device transmits the data block received from the upstream device to the downstream device according to the communication descriptor in the data block. Furthermore, the current device can obtain the corresponding task template according to the communication descriptor in the data block, and determine the task processing information as task forwarding information according to the task template, and the task forwarding information is used to indicate that the data transmitted by the upstream device needs to be transmitted to its downstream device. Among them, the task template is used to configure the processing information of the communication slice received by the current device.

[0117] Optionally, the above-mentioned communication descriptor may include a first communication descriptor and a second communication descriptor, wherein the first communication descriptor is used to represent the communication descriptor corresponding to the first n-1 data transmissions, and the second communication descriptor is used to represent the communication descriptor corresponding to the nth data transmission. n represents the total number of data transmissions performed before each device performs data operations, n is greater than or equal to 2, and n is a positive integer. Optionally, n is equal to N-1. Correspondingly, the task template may also include a first task template and a second task template, wherein the first task template is used to configure the control information for forwarding data received by the receiving device of the device to the downstream device, and the first task template may include a forwarding flag and a forwarding path configured by software (the forwarding path may include the identifier of the downstream device, etc.). Optionally, there may be more than one downstream device of each device, and the device may determine which downstream devices each device needs to send data to through the downstream device identifier in its first task template. The second task template is used to configure the configuration information for the artificial intelligence processing device of the device to perform operations.

[0118] Optionally, the operation of the current device automatically forwarding the data block to a downstream device of the current device may include:

[0119] The current device transmits the data block received from the upstream device to the downstream device according to the first communication descriptor in the data block transmitted by the received upstream device.

[0120] Specifically, the task scheduling device can determine that the task processing information is task forwarding information based on the first communication descriptor it receives, so that the task scheduling device can automatically trigger the sending device of the device to send the received communication slice to the downstream device based on the task forwarding information.

[0121] like Figure 7 As shown, the operation of the current device automatically forwarding the data block to the downstream device of the current device may include:

[0122] S710, the current device obtains a first task template according to a first communication descriptor received and transmitted by an upstream device;

[0123] Among them, the first communication descriptor contains the template identifier of the first task template. The task scheduling device of the current device can obtain the first task template according to the template identifier of the first task template in the first communication descriptor. The first task template may include information such as the forwarding flag and forwarding path configured by software (the forwarding path may include the identifier of the downstream device, etc.).

[0124] S720: The current device obtains task forwarding information of the current device according to the first task template.

[0125] Optionally, the task forwarding information of the current device may include whether to forward and the forwarding path, etc., which are not specifically limited here. The current device can determine whether to forward according to the forwarding flag in the first task template, and determine which downstream devices to transmit the data block to according to the forwarding path in the first task template.

[0126] S730: The current device transmits the data block received from the upstream device to the downstream device according to the task forwarding information.

[0127] Optionally, the task scheduling device may also stop data transmission between devices according to the second communication descriptor it receives. The above method may also include:

[0128] The current device stops sending data blocks to the downstream device of the current device according to the second communication descriptor in the received data block transmitted by the upstream device.

[0129] Among them, the current device can determine, based on the second communication descriptor it receives, that the data block received this time is the data block of the nth data transmission (i.e., the data block of the last data transmission), and the current device can stop data transmission between devices based on the second communication descriptor, that is, the above-mentioned operations S520 and S530 have been repeated a preset number of times, and the preset conditions are met, and the sending of data blocks to the downstream device of the current device is stopped.

[0130] Furthermore, the current device may also determine the task execution information according to the second communication descriptor it receives, so as to perform calculations according to the task execution information and obtain calculation results. Figure 8 As shown, the above method may also include:

[0131] S810: The current device obtains a second task template according to a second communication descriptor received and transmitted by an upstream device.

[0132] The second communication descriptor includes a template identifier of the second task template, and the current device can obtain the second task template according to the second task template identifier in the second communication descriptor. The second task template can be configured with information required to perform the operation, including but not limited to the storage address and size of the kernel function (kernel), and related parameter information required for the operation, etc.

[0133] S820: The current device obtains task execution information of the current device according to the second task template.

[0134] S830: The current device performs calculations according to the task execution information of the current device and the data blocks stored in the preset storage space to obtain calculation results.

[0135] Optionally, the task scheduling device of the current device triggers the start-up of the artificial intelligence processing device of the current device according to the task execution information of the current device; the artificial intelligence processing device of the current device performs calculations according to the N communication slices stored in the preset storage space to obtain calculation results.

[0136] Optionally, before the current device performs calculations according to the task execution information, the current device may also determine whether the data required for the calculation is complete. When it is determined that the data required for the calculation is complete, the current device may perform calculations according to the task execution information of the current device and the data blocks stored in the preset storage space to obtain calculation results. Optionally, the current device may send an interrupt signal to the host, and the host may return a feedback signal to the current device indicating whether the data is complete, and the current device may determine whether to perform the calculation operation according to the feedback signal. Further, optionally, after the current device completes the calculation operation and obtains the calculation result, it may also send a calculation completion signal to the host device.

[0137] Optionally, the task template (including the first task template and the second task template) can be configured by the driver to each device before data is transmitted between the devices. Furthermore, the task template can be configured to the task scheduling device of each device. Specifically, the driver can send the task template to each device respectively, each device can return the template identifier of each task template to the driver, and the driver can write the template identifier of each task template into the corresponding communication descriptor.

[0138] Optionally, the communication descriptor can be configured by the driver before data is transmitted between devices. Specifically, the driver can send Rx tasks to each device respectively, and after each device receives the Rx task, the downstream device can transmit its corresponding task template identifier and the address of the preset storage space corresponding to the downstream device to its upstream device. Afterwards, the driver can also send Tx tasks to each device respectively, and configure the communication descriptor to the current device according to the task template identifier transmitted by the downstream device of the current device and the address of the preset storage space corresponding to the downstream device.

[0139] Optionally, when the size of a data block is greater than a preset threshold, when each device receives a data block transmitted by an upstream device, the task scheduling device on each device may trigger the start of its artificial intelligence processing device, and the artificial intelligence device may perform the above-mentioned related operations based on the data block it receives, and store the operation results in the second storage space. In the embodiment of the present application, since the data block is relatively large, the time it takes to transmit the data block from one device to another is equivalent to the time it takes for the artificial intelligence processing device to perform the operation. At this time, each device can start the artificial intelligence device to perform the operation after receiving the data block, which can reduce the processing delay on the one hand, and avoid storing N data blocks on the device, thereby reducing the storage space occupied by the device.

[0140] Optionally, to further improve processing efficiency, when the artificial intelligence processing device of each device is performing calculations, the sending device or receiving device on each device can simultaneously transmit data. When the artificial intelligence processing device of each device is performing calculations, the data block can also be moved at the same time. Through the parallel operation of different devices on the device, a pipeline operation mode of various processing operations can be realized, thereby reducing processing delays and improving processing efficiency.

[0141] The embodiment of the present application further provides a computer storage medium, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the method described in any of the above embodiments can be implemented. When the processor executes the computer program in the memory, the following steps can be implemented:

[0142] Receiving a data block transmitted by an upstream device, and storing the data block in a preset storage space;

[0143] Transmitting the data block received from the upstream device to the downstream device of the current device;

[0144] The above operations are repeatedly performed until a preset condition is met, and when the preset condition is met, calculations are performed according to the data blocks stored in the preset storage space to obtain calculation results.

[0145] It should be clear that the process of the processor executing the computer program in the embodiment of the present application is basically consistent with the various execution steps of the method in the above embodiment. For details, please refer to the description above and will not be repeated here.

[0146] Among them, any reference to memory, storage, database or other media used in the embodiments provided in the present application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0147] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0148] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A task processing method, characterized in that: The method is applied to a system including N devices, wherein N is greater than or equal to 2; a preset storage space is provided on each device, and the size of the preset storage space is greater than or equal to the size of N data blocks of the N devices; the method comprises: Receiving a data block transmitted by an upstream device, and storing the data block in a preset storage space of the current device; wherein the data block carries a first communication descriptor and a second communication descriptor; Automatically transmitting a data block received from an upstream device of the current device to a downstream device of the current device according to a first communication descriptor in a received data block transmitted by the upstream device; Repeat the above steps until a preset condition is met, and when the preset condition is met, perform calculations according to the data blocks stored in the preset storage space to obtain calculation results; The method further includes: stopping sending data blocks to a downstream device of the current device according to a second communication descriptor in a received data block transmitted by the upstream device.

2. The method according to claim 1, characterized in that The method further comprises: When the above operations are repeatedly performed for a preset number of times, it is determined that the preset condition is satisfied.

3. The method according to claim 1, characterized in that Before receiving the data block transmitted by the upstream device, the method further includes: Sending a data block to a downstream device of the current device according to a received data transmission trigger signal; The data transmission trigger signal is sent by a driver to the N devices.

4. The method according to claim 1, characterized in that: The method comprises: transmitting the data block received from the upstream device to the downstream device according to the first communication descriptor in the received data block transmitted by the upstream device, comprising: Obtaining a first task template according to a first communication descriptor received and transmitted by the upstream device; According to the first task template, obtaining task forwarding information of the current device; The data block received from the upstream device is transmitted to the downstream device according to the task forwarding information.

5. The method according to claim 4, characterized in that The method further comprises: Obtaining a second task template according to the received second communication descriptor transmitted by the upstream device; According to the second task template, obtaining task execution information of the current device; An operation is performed according to the task execution information of the current device and the data block stored in the preset storage space to obtain an operation result.

6. The method according to claim 5, characterized in that The first task template and the second task template are configured for each of the devices by a driver.

7. The method according to claim 5, characterized in that The current device includes a task scheduling device and an artificial intelligence processing device; the operation is performed according to the task processing information of the current device and the data block stored in the preset storage space to obtain the operation result, including: The task scheduling device of the current device triggers the artificial intelligence processing device of the current device to start according to the task execution information of the current device; The artificial intelligence processing device of the current device performs calculations according to the data blocks stored in the preset storage space to obtain calculation results.

8. The method according to claim 1, characterized in that The method further comprises: The calculation result is stored from the preset storage space to the cache space.

9. The method according to claim 1, characterized in that: The first communication descriptor and the second communication descriptor are configured by a driver; The preset storage space of the current device is allocated by a driver.

10. The method according to claim 1, characterized in that The size of the data block is less than or equal to a preset threshold.

11. A system, characterized in that: The method comprises N devices connected to each other, wherein the N devices are used to send data blocks to downstream devices connected thereto respectively; wherein N is greater than or equal to 2; wherein each device is provided with a preset storage space, and the size of the preset storage space is greater than or equal to the size of the N data blocks of the N devices; The current device is used to receive a data block transmitted by an upstream device, and store the data block in a preset storage space of the current device, and automatically transmit the data block received from the upstream device of the current device to the downstream device according to the first communication descriptor in the data block transmitted by the upstream device, and then repeat the above operation until a preset condition is met; wherein the data block carries the first communication descriptor and the second communication descriptor; The current device is further configured to perform a calculation based on the data block stored in the preset storage space to obtain a calculation result when a preset condition is met; The current device is further configured to stop sending data blocks to a downstream device of the current device according to a second communication descriptor in a received data block transmitted by the upstream device.

12. The system according to claim 11, characterized in that Each of the devices includes a communication module and a task scheduling device, and the communication module includes: A receiving device, configured to receive a data block transmitted by an upstream device; and Transmitting means, for transmitting a data block received from the upstream device to a downstream device of the device; The task scheduling device is used to determine task processing information according to the communication data transmitted by the receiving device.

13. The system according to claim 12, characterized in that The task scheduling device is also used to obtain a first task template according to the received first communication descriptor, and determine task forwarding information according to the first task template to trigger the sending device to transmit the data block received from the upstream device to the downstream device of each of the devices.

14. The system according to claim 12, characterized in that Each of the devices also includes artificial intelligence processing means; The task scheduling device is also used to obtain a second task template based on the received second communication descriptor, and determine the task processing information based on the second task template to trigger the artificial intelligence processing device to perform calculations based on the task processing information of the current device and the data blocks stored in the preset storage space to obtain calculation results.

15. A computer storage medium, characterized in that The method comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 10 can be implemented.

Images