A data transmission method and apparatus

Abstract

The present application provides a data transmission method and apparatus for improving the data transmission efficiency between coprocessors, so as to match the computing power of the AI system. This method is applied to a computer system including at least two coprocessors, such as including a first coprocessor and a second coprocessor. A shared memory is deployed between the first coprocessor and the second coprocessor for storing the data generated by their respective execution of subtasks. Further, the shared memory also stores the storage addresses of the data generated by the execution of subtasks, the mapping relationships between each subtask and the coprocessor executing the subtask. Thus, the storage address of the data to be read by the coprocessor can be found through this mapping relationship, and then the data can be directly read from the shared memory without copying the data through the system bus, improving the data transmission efficiency between coprocessors.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of communication technologies, and in particular, to a data transmission method and apparatus. Background Art

[0002] Artificial intelligence (AI) has made significant breakthroughs in application fields such as speech recognition and computer vision. This has also prompted traditional AI systems composed of central processing units (CPUs) to be replaced by AI systems composed of CPUs + accelerators (such as graphics processing units (GPUs) and / or neural-network processing units (NPUs)). In an AI system, a connection is established between the CPU and the accelerator through a system bus. If an accelerator needs to obtain data from another accelerator, multiple data copies need to be performed through the system bus with the help of the CPU. For example, first copy the data from one accelerator to the CPU, and then from the CPU to another accelerator. Since the growth rate of the bandwidth supported by the system bus is relatively slow, the data transmission efficiency between accelerators cannot match the improvement of the computing power (i.e., computing ability) of the AI system.

[0003] Therefore, achieving high-speed data transmission to match the growth of the computing power of the AI system and thereby improving the overall performance of the AI system is an urgent problem to be solved. Summary of the Invention

[0004] Embodiments of the present application provide a data transmission method and apparatus to improve data transmission efficiency so as to match the computing power of the AI system.

[0005] The method and the apparatus applying the method are introduced below through different aspects. The implementation manners and beneficial effects of the following aspects can be referred to each other.

[0006] In a first aspect, the present application provides a data transmission method, which is applied to a computer system including at least two coprocessors. For example, it includes a first coprocessor and a second coprocessor. When the first coprocessor needs to obtain first data generated by the second coprocessor during the execution of a second subtask during the execution of a first subtask, it determines a first storage address for storing the first data according to a mapping relationship, where the mapping relationship includes the association relationship between the identifier of the second coprocessor, the identifier of the second subtask executed by the second coprocessor, and the first storage address; then the first coprocessor can read the first data from a first storage space corresponding to the first storage address; wherein, the first storage space is located in the shared memory of the first coprocessor and the second coprocessor.

[0007] The coprocessor involved in this application (such as the above-mentioned first coprocessor or second coprocessor) is usually an acceleration device for acceleration. The acceleration device includes a processor for acceleration (abbreviated as accelerator). The accelerator can be one or more of a GPU, NPU, field programmable gate array (FPGA), application specific integrated circuit (ASIC), or digital signal processor (DSP). The accelerator usually belongs to a dedicated processor. The coprocessor provided in this application also includes general-purpose processors such as a CPU, a microcontroller unit (MCU), and an application processor (AP).

[0008] In the prior art, when accelerators interact with each other, data is copied from the memory of one accelerator to the memory of another accelerator through the system bus, resulting in the overall performance of the AI system depending on the transmission capacity of the system bus. In this application, by deploying a shared memory between coprocessors, the shared memory is used to store the data generated by each coprocessor when executing subtasks, which can also be said to be the data that needs to be interacted between coprocessors (or between accelerators). The storage addresses of the data generated by each coprocessor when executing subtasks can be found through the mapping relationship. Based on this, the data of the second coprocessor that the first coprocessor wants to read can determine the storage address according to the mapping relationship, and then can directly read the data from the shared memory according to the storage address of the data. There is no need to copy data through the system bus, which improves the data transmission efficiency between coprocessors.

[0009] In a possible design, the above-mentioned first coprocessor may include a first processor and a second processor; the first processor may be the above-mentioned accelerator. The second processor may be the above-mentioned general-purpose processor. The first processor in the first coprocessor executes the first subtask. During the execution of the first subtask by the first processor, a first acquisition request is sent to the second processor. The first acquisition request is used to request to acquire the first data generated by the second coprocessor during the execution of the second subtask; then after the second processor receives the first acquisition request, it determines the first storage address of the first storage space for storing the first data according to the mapping relationship, and can send the first storage address to the first processor; finally, the first processor reads the first data from the first storage space corresponding to the first storage address.

[0010] In a possible design, the first coprocessor may include a task schedule engine (TSE). The task schedule engine stores program code; when the first coprocessor executes the first storage address for storing the first data determined according to the mapping relationship, it may execute the first storage address of the first storage space for storing the first data determined according to the mapping relationship by running the program code stored in the task schedule engine.

[0011] In an example, when the first coprocessor includes a first processor and a second processor, the second processor may run the program code stored in the task schedule engine to execute the first storage address of the first storage space for storing the first data determined according to the mapping relationship.

[0012] In another example, the first coprocessor includes a first processor and a task schedule engine, the task schedule engine includes a second processor, and program code is stored. The second processor runs the program code to implement the execution of the first storage address of the first storage space for storing the first data determined according to the mapping relationship. Alternatively, the task schedule engine is implemented by a hardware circuit.

[0013] In yet another example, the first coprocessor includes a first processor, a second processor, and a task schedule engine. A processor is also deployed in the task schedule engine, which can be called the third processor for the sake of distinction. The first storage address of the first storage space for storing the first data determined according to the mapping relationship is executed in cooperation with the second processor and the third processor.

[0014] It should be noted that this application does not limit the number of the first processor and the second processor included in the first coprocessor.

[0015] In a possible design, the above mapping relationship may be stored in a shared memory. The mapping relationship may be pre-established. If it is necessary to store the data generated by a coprocessor executing a subtask, it may be stored according to the mapping relationship.

[0016] For example, during the execution of the first subtask by the first coprocessor, second data is generated. If it is necessary to store the second data, the second storage address for storing the second data can be determined according to the mapping relationship. The mapping relationship includes the association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor, and the second storage address; the second storage address corresponds to a second storage space in the shared memory, and the second storage space is used to store the second data generated during the execution of the first subtask by the first coprocessor; further, the first coprocessor can store the second data in the second storage space corresponding to the second storage address. Therefore, when other coprocessors need to obtain the second data, they can obtain the second storage address of the second storage space for storing the second data according to the mapping relationship, so that there is no need to copy data through the system bus, improving the data transmission efficiency between coprocessors.

[0017] In a possible design, the first coprocessor can run the program code in the task scheduling engine to execute the determination of the second storage address for storing the second data according to the mapping relationship.

[0018] In a possible design, the above mapping relationship can be established during the execution of the subtask by the coprocessor. After storing the data of the subtask, a mapping relationship item can be added according to the storage address of the data. For example, during the execution of the first subtask by the first coprocessor, second data is generated, a second storage space is determined in the shared memory, and the second data is stored in the second storage space, and the second storage space corresponds to the second storage address; then, the association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor, and the second storage address is added to the mapping relationship. Therefore, when other coprocessors need to obtain the second data, they can obtain the second storage address of the second storage space for storing the second data according to the mapping relationship, so that there is no need to copy data through the system bus, improving the data transmission efficiency between coprocessors.

[0019] In a possible design, the first coprocessor can run the program code in the task scheduling engine to execute the addition of the association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor, and the second storage address to the mapping relationship.

[0020] In a possible design, the first coprocessor receives the first subtask sent by the main processor and starts to execute the first subtask. The above-mentioned coprocessors (including the first coprocessor and the second coprocessor) are included in the AI system. The AI system may also include a main processor. Here, the main processor can be a CPU. The main processor can be used to split tasks and send the split subtasks to the coprocessor, so that the coprocessor assists the main processor in executing tasks to improve the computing power of the AI system. In the above design, the main processor sends a subtask to the coprocessor, which triggers the coprocessor to execute the subtask.

[0021] In a possible design, when the main processor sends a subtask to the coprocessor, it can send the start condition of the subtask to the coprocessor. Taking the first coprocessor as an example, the first coprocessor receives the first subtask sent by the main processor and the start condition of the first subtask; when the start condition of the first subtask is met, the first coprocessor starts to execute the first subtask. In the above design, the main processor sends the start condition to the coprocessor, so that the coprocessor determines whether the start condition is met. When it is determined that the start condition is met, the subtask is then started and executed.

[0022] The start condition can also be referred to as the start timing. For example, the start condition can include, but is not limited to, one or more of the following conditions:

[0023] 1) The duration of receiving the subtask reaches a preset duration.

[0024] 2) A preset time point is reached.

[0025] 3) The timer corresponding to the task start times out.

[0026] 4) The previous subtask executed serially is completed.

[0027] In a second aspect, the present application provides a coprocessor (such as the first coprocessor or the second coprocessor in the foregoing first aspect). This coprocessor can also be considered as an independent chip or a partial structure in a chip. In some scenarios, this coprocessor can also be referred to as a coprocessing device or an acceleration device, etc.

[0028] Since there are multiple coprocessors, for the convenience of description, this coprocessor is referred to as the first coprocessor in this aspect. Specifically, the first coprocessor includes at least a first processor and a second processor. The first processor can be used to execute subtasks. The second processor can be used to manage the first processor. For example, the first processor can be an accelerator; the second processor can be a CPU or other types of general-purpose processors.

[0029] A first processor, when it is necessary to obtain first data generated by a second coprocessor during the execution of a second subtask during the execution of a first subtask, sends a first acquisition request to the second processor. The first acquisition request is used to request the acquisition of the first data generated by the second coprocessor during the execution of the second subtask; A second processor, configured to determine a first storage address of a first storage space for storing the first data according to a mapping relationship, and send the first storage address to the first processor; wherein, the mapping relationship includes an association relationship between an identifier of the second coprocessor, an identifier of the second subtask executed by the second coprocessor, and the first storage address; The first processor is configured to read the first data from the first storage space corresponding to the first storage address; wherein, the first storage space is located in a shared memory of the first coprocessor and the second coprocessor.

[0030] In a possible design, the first coprocessor may further include a task scheduling engine. The task scheduling engine stores program code. The above-mentioned second processor is specifically configured to run the program code stored in the task scheduling engine to execute determining a first storage address of a first storage space for storing the first data according to the mapping relationship, and sending the first storage address to the first processor. In some other possible designs, the task scheduling engine may also be implemented by a hardware circuit.

[0031] In a possible design, when the first processor is further configured to store second data generated during the execution of the first subtask, it sends a second acquisition request to the second processor. The second acquisition request is used to request a storage space for storing the second data;

[0032] The second processor is further configured to, after receiving the second acquisition request, determine a second storage address according to the mapping relationship, and send the second storage address to the first processor;

[0033] wherein, the mapping relationship further includes an association relationship between an identifier of the first coprocessor, an identifier of the first subtask executed by the first coprocessor, and the second storage address; the second storage address corresponds to a second storage space, which is used to store the second data generated during the execution of the first subtask by the first coprocessor;

[0034] The first processor is further configured to store the second data in the second storage space corresponding to the second storage address.

[0035] In a possible design, the second processor is specifically configured to run the program code stored in the task scheduling engine to execute determining the second storage address according to the mapping relationship, and sending the second storage address to the first processor.

[0036] In a possible design, when the first processor is further used to store the second data generated during the execution of the first subtask, it sends a third acquisition request to the second processor, and the third acquisition request is used to request the storage space for storing the second data.

[0037] The second processor is further configured to, after receiving the second acquisition request, allocate a second storage space for the first processor. The second storage space corresponds to a second storage address, notify the first processor of the second storage address, and add an association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor, and the second storage address in the mapping relationship.

[0038] The first processor is further configured to store the second data in the second storage space corresponding to the second storage address.

[0039] In a possible design, the second processor is specifically configured to run the program code stored in the task scheduling engine to execute adding an association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor, and the second storage address in the mapping relationship.

[0040] In a possible design, the second processor is further configured to receive the first subtask sent by the main processor and send the first subtask to the first processor. The first processor is further configured to, when receiving the first subtask, start executing the first subtask.

[0041] In a possible design, the second processor is further configured to receive the first subtask and the start condition of the first subtask sent by the main processor, send the first subtask to the first processor, and when it is determined that the start condition of the first subtask is met, trigger the first processor to execute the first subtask. The first processor is further configured to receive the first subtask and start executing the first subtask under the trigger of the second processor.

[0042] For the beneficial effects of the second aspect and any design of the second aspect, reference can be made to the beneficial effects of the first aspect and any design of the first aspect, which will not be elaborated here.

[0043] In a third aspect, the present application provides a computer system. The computer system can also be referred to as an AI system. The computer system includes at least a first coprocessor and a second coprocessor. The computer system may include other coprocessors. The present application does not limit the number of coprocessors included in the computer system.

[0044] Specifically, a first coprocessor is configured to execute a first subtask; a second coprocessor is configured to execute a second subtask; the first coprocessor is further configured to, when it is necessary to obtain first data generated by the second coprocessor during the execution of the second subtask during the execution of the first subtask, determine a first storage address for storing the first data according to a mapping relationship, and read the first data from a first storage space corresponding to the first storage address. The mapping relationship includes an association relationship between an identifier of the second coprocessor, an identifier of the second subtask executed by the second coprocessor, and the first storage address; wherein, the first storage space is located in a shared memory of the first coprocessor and the second coprocessor.

[0045] The first coprocessor and the second coprocessor may be deployed in one chip or in different chips. When deployed in the same chip, the computer system may also be understood as the chip system.

[0046] In a possible design, the first coprocessor is further configured to generate second data during the execution of the first subtask, and determine a second storage address according to the mapping relationship; wherein, the mapping relationship further includes an association relationship between an identifier of the first coprocessor, an identifier of the first subtask executed by the first coprocessor, and the second storage address; the second storage address corresponds to a second storage space, and the second storage space is used to store the second data generated by the first coprocessor during the execution of the first subtask; store the second data in the second storage space corresponding to the second storage address.

[0047] In a possible design, the first coprocessor is further configured to generate second data during the execution of the first subtask, store the second data in a second storage space, and the second storage space corresponds to a second storage address; add an association relationship between an identifier of the first coprocessor, an identifier of the first subtask executed by the first coprocessor, and the second storage address to the mapping relationship.

[0048] In a possible design, the computer system further includes a main processor. The main processor is configured to split a task to be processed, for example, split it into two subtasks, namely a first subtask and a second subtask, so that the main processor issues the first subtask to the first coprocessor and issues the second subtask to the second coprocessor.

[0049] In a possible design, the main processor may be deployed in a different chip from the first coprocessor and the second coprocessor. For example, the computer system includes two chips, one chip includes the main processor, and the other chip includes at least two coprocessors, such as the first coprocessor and the second coprocessor described above.

[0050] In a possible design, the main processor is further configured to send the start condition of the first subtask to the first coprocessor and the start condition of the second subtask to the second coprocessor;

[0051] The first coprocessor is specifically configured to start executing the first subtask when it determines that the start condition of the first subtask is met;

[0052] The second coprocessor is specifically configured to start executing the second subtask when it determines that the start condition of the second subtask is met.

[0053] In a possible design, the first coprocessor includes a first processor and a second processor. The first processor is used to execute subtasks, and the second processor is used to manage the first processor.

[0054] The second processor is used to receive the first subtask from the main processor and the start condition of the first subtask, and send the first subtask to the first processor; the second processor is further used to trigger the first processor to execute the first subtask when it determines that the start condition of the first subtask is met; the first processor is used to execute the first subtask under the trigger of the second processor.

[0055] In the above design, at least two processors are provided in the coprocessor, one processor has a control function, and other processors are used to execute subtasks. It can reduce the interaction with the main processor and improve the communication performance.

[0056] In a possible design, the first processor is further configured to perform the operations performed by the first processor in the second aspect or any design of the second aspect. The second processor is further configured to perform the operations performed by the second processor in the second aspect or any design of the second aspect. For details, refer to the description in the second aspect or any design of the second aspect, which will not be elaborated here.

[0057] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes: computer program code, when the computer program code is run by a coprocessor, it causes the coprocessor to execute any method in the first aspect or any possible implementation manner of the first aspect.

[0058] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, and the computer-readable storage medium stores a program, and the program causes the coprocessor to execute any method in the first aspect or any possible implementation manner of the first aspect.

[0059] In a sixth aspect, an embodiment of the present application provides a computer program, when the computer program is executed on a certain computer, it will cause the computer to implement any method in the first aspect or any possible implementation manner of the first aspect.

[0060] In a seventh aspect, an embodiment of the present application provides a chip system, which includes one or more coprocessors as described in the second aspect or any possible design of the second aspect.

[0061] For the technical effects that can be achieved by the second aspect to the seventh aspect, reference can be made to the above analysis and description of the first aspect and each possible design in the first aspect, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the implementation method of GPU direct technology in an embodiment of the present application;

[0063] Figure 2 It is a schematic diagram of the implementation method of NVlink technology in an embodiment of the present application;

[0064] Figure 3 It is a schematic diagram of the implementation method of Direct GPU technology in an embodiment of the present application;

[0065] Figure 4 It is a schematic diagram of a computer system architecture in an embodiment of the present application;

[0066] Figure 5 It is a schematic diagram of another computer system architecture in an embodiment of the present application;

[0067] Figure 6 It is a schematic diagram of the data transmission method flow in an embodiment of the present application;

[0068] Figure 7 It is a schematic diagram of yet another computer system architecture in an embodiment of the present application;

[0069] Figure 8 It is a schematic diagram of the neural network task execution process in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0070] First, some terms in the present application are explained to facilitate understanding by those skilled in the art.

[0071] 1) Shared virtual memory (SVM).

[0072] The way of shared virtual memory is to map a physical memory to the virtual address spaces of different processes respectively, so that each process can read the same data, thereby realizing process communication. Because communication is achieved through memory operations, it is a most efficient data exchange method.

[0073] 2) Process.

[0074] A process is an execution activity of a program in a computer with respect to a certain data set. It is the basic unit for the system to allocate and schedule resources and is the foundation of the operating system structure. A process is the basic execution entity of a program, and a program is a description of instructions, data, and their organizational forms.

[0075] 3) Cache coherence. A cache is also known as a cache memory. Cache coherence refers to the mechanism that ensures the data in the cache memory is the same as the data in the main memory in a computer system with a hierarchical storage system.

[0076] To improve the data transfer efficiency between accelerators in existing AI systems, there are two possible solutions. One solution is to adopt GPU direct and NVlink technologies, and the other solution is to adopt Direct GPU / FPGA technology. The accelerators used in the AI systems deployed with GPU direct and NVlink technologies are GPUs. The accelerators used in the AI systems deployed with Direct GPU / FPGA technology are GPUs and FPGAs.

[0077] Before the GPU direct technology, as an independent device, to complete the communication between GPUs, 3 data copies were required. As shown in (a) in Figure 1 , the 3 data copies are as follows: The CPU first copies the memory data of GPU1 to system cache 1, then copies it from system cache 1 to system cache 2, and finally copies it to the memory of the peer GPU2. System cache 1 corresponds to GPU1, and system cache 2 corresponds to GPU2. This frequent data copying greatly limits the computing power of the GPU because the GPU needs to spend a long time waiting for the peer to copy the data over. In the GPU direct technology, multiple GPUs are directly connected to the CPU through the peripheral component interconnect express (PCIe). The GPU supports sharing the memory on the CPU side through PCIe, thus realizing data communication between GPUs. As shown in (b) in Figure 1 , GPU direct only requires 2 data copies: The CPU first copies from the memory of the GPU to the system cache, and then copies from the system cache to the memory of the peer GPU through PCle. GPU direct improves the data transfer rate through the peripheral component interconnect express bus.

[0078] There are inherent two-way bandwidth limitations in PCIe. To improve the data transfer rate between multiple GPUs, the NVlink technology emerged. The NVlink technology is a bus and its transmission protocol with a bandwidth higher than that of PCIe, and the bandwidth provided by NVlink can reach 20 GB / s. See Figure 2 As shown, the NVlink technology adds a two-way NVlink bus channel between GPUs, and data copying between GPUs can be achieved through the NVlink bus channel, thus eliminating the need to copy data through the CPU.

[0079] In the Direct GPU / FPGA technology, the GPU and the FPGA are connected to the CPU through the PCIe bus, and data exchange between the GPU and the FPGA is achieved through the PCIe bus. See Figure 3 As shown, when the GPU needs to copy data from the double data rate memory (DDR memory) of the FPGA, it can directly copy the data from the DDR memory of the FPGA through the PCIe bus without going through the CPU.

[0080] As can be seen from the above, in these two technologies, although the computing power of accelerators (such as GPUs or other types of acceleration devices) can be improved to a certain extent, both of the above solutions require data exchange between accelerators to be achieved through the system bus (such as the PCIe bus or the NVlink bus). However, the bandwidth supported by the system bus is limited, and the computing power of the AI system is still restricted by the bandwidth supported by the system bus, which further leads to the inability of the data transfer capacity between accelerators to match the growth of the computing power of the AI system.

[0081] Based on this, the embodiments of the present application provide a data transfer method and apparatus, specifically adopting a direct connection solution between accelerators to bypass the limitation of the bus on the computing power of the accelerators.

[0082] See Figure 4 As shown, it is a schematic diagram of a computer system architecture provided by the embodiments of the present application. The computer system includes a main processor and M coprocessors, where M is an integer greater than 1. The coprocessors involved in the embodiments of the present application at least include accelerators. The main processor and the coprocessors are located in different chips. The M coprocessors can be deployed in different chips, or can be deployed in the same chip, or the M coprocessors can also be deployed in X chips, where X is an integer less than M and greater than 1. The main processor and the coprocessors are connected through the system bus. The system bus is responsible for communication between devices. For example, it can be PCIe, or other types of buses, and the embodiments of the present application do not make specific limitations on this. Exemplarily, Figure 4In the system, the main processor and the coprocessors, as well as between each coprocessor, are connected through a PCIe switch chip. The PCIe switch chip is a chip that provides the conversion (switch) of PCIe connections. The processor and the coprocessor can be connected through a fast bus that ensures cache coherence. The fast bus can be the Huawei cache coherent system (HCCS), the advanced eXtensible interface (AXI), the Hyper transport and quick path interconnect (Hyper transport and QPI), or the advanced microcontroller bus architecture (AMBA). Figure 4 Taking M = 4, that is, 4 coprocessors as an example, they are coprocessor 1 - coprocessor 4 respectively.

[0083] Each coprocessor corresponds to a globally unique device identifier (ID) in the computer system. This device ID can be achieved through configuration, or can be assigned by the main processor to each coprocessor. For example, after the computer system starts, the main processor assigns device IDs to each coprocessor, and synchronizes the device IDs corresponding to each coprocessor to each coprocessor through the system bus. That is, the main processor sends the device IDs corresponding to coprocessor 1 - coprocessor 4 to coprocessor 1 - coprocessor 4 respectively, so that each coprocessor can obtain the device IDs of each coprocessor in the computer system.

[0084] After obtaining a computing task, the main processor divides the computing task into N subtasks, where N is less than or equal to M. In one example, these N subtasks are executed in parallel, then the main processor selects N coprocessors according to the device IDs of each coprocessor, and distributes the N subtasks to the N coprocessors for processing respectively. In another example, some of the N subtasks are executed in parallel, and some subtasks need to be executed after other subtasks are completed, that is, serially executed. In this case, several coprocessors can be selected for processing according to requirements and distributed to the selected coprocessors respectively.

[0085] There is shared memory among N coprocessors. The shared memory can be used to store data generated by the coprocessors during the execution of subtasks. In one way, the shared memory can be configured for each coprocessor by the main processor using the SVM technology. In another way, there is a shared memory among the coprocessors. The shared memory includes the above-mentioned shared memory. The shared memory can be implemented by one or more memories. In one example, one or more memories can be set in the computer system independently of each coprocessor. In another example, there are memories in each coprocessor, and the memories included in each coprocessor are connected by a fast bus that ensures cache coherence, and are shared through the SVM technology to form the shared memory.

[0086] During the execution of subtasks, the coprocessor saves the data generated during the execution of the subtask in the shared memory. When a coprocessor needs the data generated by another coprocessor during the execution of a subtask during its own execution of a subtask, it can obtain the storage address of the data of the subtask processed by the device ID of the other coprocessor according to the mapping relationship, and then read the data from the storage space corresponding to the storage address. When the coprocessor reads the data from the storage space corresponding to the storage address, it can be implemented using direct memory access (DMA) technology, or other data reading technologies. For the relevant description of the DMA technology, refer to the prior art, and this application will not elaborate.

[0087] The above mapping relationship includes the device ID of each coprocessor, the identifier of the subtask executed by each coprocessor, and the storage address of the storage space for storing the data during the execution of the subtask by each coprocessor. The mapping relationship can be saved in the shared memory. The storage address of the storage space where the mapping relationship is saved can be notified to each coprocessor when configuring the mapping relationship. The specific configuration method and maintenance method of the mapping relationship will be described in detail later, and will not be elaborated here.

[0088] Taking coprocessor 1 for executing subtask 1 and coprocessor 2 for executing subtask 2 as an example. Taking the case where coprocessor 1 needs to obtain the first data generated by coprocessor 2 during the execution of subtask 2 during the execution of subtask 1 by coprocessor 1. For example, the first data is stored in the first storage space of the shared memory. The storage address of the first storage space is the first storage address. The mapping relationship includes the association relationship between the device ID of coprocessor 2, the identifier of subtask 2 executed by coprocessor 2, and the first storage address. Furthermore, coprocessor 1 can determine the first storage address for storing the first data according to the mapping relationship, and read the first data from the first storage space corresponding to the first storage address. The first storage space is the storage space in the shared memory.

[0089] The following describes the specific configuration method and maintenance method of the mapping relationship.

[0090] The mapping relationship can be configured in the form of a table, or in the form of an array, or in other ways. This application does not make specific limitations on this. When configuring later, the case of configuring the mapping relationship in the form of a table is taken as an example for description. For the convenience of description, the table obtained by configuring the mapping relationship is called the address information table. When configuring the mapping relationship in the form of a table, the mapping relationship can be configured in the form of one or more tables, that is, the address information table can be composed of one or more tables.

[0091] For example, coprocessor 1 executes subtask 1, coprocessor 2 is used to execute subtask 2, coprocessor 3 is used to execute subtask 3, and coprocessor 4 is used to execute subtask 4. The device ID of coprocessor 1 is ID1, the device ID of coprocessor 2 is ID2, the device ID of coprocessor 3 is ID3, and the device ID of coprocessor 4 is ID4. The identifier of subtask 1 is A1, the identifier of subtask 2 is A2, the identifier of subtask 3 is A3, and the identifier of subtask 4 is A4.

[0092] In one example, the mapping relationship is configured in the form of one table, that is, the address information table consists of one table. As shown in Table 1, Table 1 is used to describe the association relationship between the coprocessor device ID, the identifier of the subtask executed by each coprocessor, and the storage address of the data generated during the execution of the subtask. The storage address of the data here can be the physical address of the storage space for storing the data, or the virtual address corresponding to the physical address, or can include the association relationship between the virtual address and the physical address of the data. The association relationship between the virtual address and the physical address is the relationship described by the address translation table. In other words, the mapping relationship is used to describe the association relationship between the coprocessor device ID, the identifier of the subtask executed by each coprocessor, and the address translation table corresponding to the data generated during the execution of the subtask. The establishment of the address translation table can be static configuration or dynamic generation, which is similar to the existing allocation and mapping of virtual addresses and physical addresses, and this application does not describe it in detail.

[0093] Table 1

[0094]

[0095] In another example, the mapping relationship is configured in the form of multiple tables. Taking the implementation with two tables as an example, as shown in Table 2, Table 2 is used to describe the association relationship between the coprocessor device ID and the identifier of the subtask executed by each coprocessor. As shown in Table 3, Table 3 is used to describe the address translation table corresponding to the identifier of each subtask. The address translation table is used to describe the mapping relationship between the virtual address and the physical address.

[0096] Table 2

[0097] Device ID Sub-task identifier ID1 A1 ID2 A2 ID3 A3 ID4 A4

[0098] Table 3

[0099]

[0100] In another example, as shown in Table 4, Table 4 is used to describe the association relationship between the coprocessor device ID, the identification of the subtasks executed by each coprocessor, and the storage address for storing the address translation table. As shown in Table 5, it is used to describe the address translation table corresponding to the data stored on each coprocessor.

[0101] Table 4

[0102] Device ID Sub-task identifier Storage address ID1 A1 Address 1 ID2 A2 Address 2 ID3 A3 Address 3 ID4 A4 Address 3

[0103] Table 5

[0104]

[0105]

[0106] In a possible implementation manner, the mapping relationship can be established by the main processor, or can be established by the coprocessor, or can be jointly established by the main processor and the coprocessor.

[0107] In one example, the mapping relationship is established by the main processor and notified to each coprocessor. For example, taking the mapping relationship shown in Table 1 as an example, after the main processor starts up and assigns device IDs and subtasks to each coprocessor, it can create the mapping relationship shown in Table 1, and then can notify each coprocessor.

[0108] In another example, the mapping relationship is established by the coprocessor. Specifically, which coprocessor is used to establish it can be indicated by the main processor or pre-configured.

[0109] In the above two examples, the mapping relationship is established before the coprocessor executes the subtasks. If the coprocessor generates data during the execution of the subtasks, the storage address for storing the data can be determined according to the mapping relationship. Here, it is also taken that coprocessor 1 is used to execute subtask 1 and coprocessor 2 is used to execute subtask 2 as an example. As mentioned above, during the execution of subtask 1 by coprocessor 1, it is necessary to obtain the first data generated by coprocessor 2 during the execution of subtask 2. Coprocessor 1 can determine the first storage address for storing the first data according to the mapping relationship, and read the first data from the first storage space corresponding to the first storage address.

[0110] If the coprocessor 1 generates second data during the execution of subtask 1, it can determine the storage address for storing the second data according to the mapping relationship.

[0111] Taking the mapping relationship shown in Table 1 as an example. The coprocessor 1 determines the storage address for storing the second data according to the mapping relationship. For example, the virtual address is V1 and the physical address is V2. V2 corresponds to the second storage space in the shared memory, that is, this second storage space is used to store the second data generated during the execution of subtask 1 by the coprocessor 1. Furthermore, the coprocessor 1 stores the second data in the second storage space corresponding to V2.

[0112] In another example, the mapping relationship is jointly established by the main processor and the coprocessor. The mapping relationship can be established and updated by the coprocessor during the execution of subtasks with the assistance of the main processor. For example, the device ID of each coprocessor and the identifier of the subtask are sent by the main processor to each coprocessor. The address translation table can be created and updated by each coprocessor during the execution of subtasks. When the mapping relationship is stored in multiple tables, a part of the multiple tables can be established by the main processor and notified to each coprocessor, and another part of the tables can be established and updated by the coprocessor during the execution of subtasks. For example, the above Table 2 or Table 4 is created by the main processor and notified to each coprocessor, and the content in Table 3 or Table 5 is created and updated by each coprocessor during the execution of subtasks.

[0113] In the above example, the mapping relationship is not established in advance before the coprocessor executes the subtask. If the coprocessor generates data during the execution of the subtask, the coprocessor can obtain the storage space in the shared memory for storing the data, then store the data in this storage space, and add the association relationship between the address of this storage space, the coprocessor, and the subtask to the mapping relationship.

[0114] Still taking the coprocessor 1 for executing subtask 1 and the coprocessor 2 for executing subtask 2 as an example.

[0115] If the coprocessor 1 generates second data during the execution of subtask 1, it determines the second storage space in the shared memory. This second storage space is a free memory block in the shared memory. Then it stores the second data in the second storage space, and this second storage space corresponds to the second storage address. For example, the second storage address corresponds to V2-P2. Thus, the coprocessor 1 can add the association relationship between the identifier (ID1) of the first coprocessor, the identifier (A1) of the first subtask executed by the first coprocessor, and the second storage address (V2-P2) to the mapping relationship.

[0116] In a possible implementation, when the coprocessor starts to execute a subtask, for example, it starts to execute under the trigger of the main processor, or it can also start to execute the subtask when it determines that the start condition is met by itself.

[0117] Taking the coprocessor 1 as an example above. In one example, the main processor sends the subtask 1 to the coprocessor 1. After receiving the subtask 1, the coprocessor 1 starts to execute the subtask. In another example, the main processor sends the subtask 1 and the start condition of the subtask 1 to the coprocessor 1. Thus, when the coprocessor 1 determines that the start condition of the subtask 1 is met, it starts to execute the first subtask.

[0118] The start condition can also be referred to as the start timing. For example, the start condition can include, but is not limited to, one or more of the following conditions:

[0119] 1) The duration of receiving the subtask reaches the preset duration.

[0120] 2) Reaching the preset time point.

[0121] 3) The timer corresponding to the task start times out.

[0122] 4) The previous subtask executed serially is completed.

[0123] In this application, each coprocessor is connected by a fast bus that ensures cache consistency. Then, through this fast bus and the mapping relationship, the data required to execute the subtask is read, without the need to copy data through the system bus or the CPU. In addition, compared with the PCIe bus, the fast bus has higher bandwidth and lower latency, which can further improve the data transmission ability of the computer system.

[0124] See Figure 5As shown in the figure, taking the example of a computer system including two coprocessors, the solution provided by the embodiments of the present application will be described in detail in combination with a specific application scenario. The computer system can also be referred to as an AI system. The main processor in the computer system may include a main processing unit and a storage unit. For example, the main processing unit can be implemented by one or more processors. For example, it is implemented by a CPU, and the CPU of the main processing unit is called the main (Host) CPU. The storage unit can be DDR. The coprocessor may include a processing unit and a storage unit. The processing unit can be implemented by one or more processors (or called accelerators). For example, it is implemented by one or more of a CPU, GPU, FPGA, NPU, or digital signal processor (DSP). The coprocessor may further include a management unit for managing the processing unit and the storage unit in the coprocessor. The management unit in the coprocessor can be implemented by one or more processors, such as a CPU. The storage unit in the coprocessor can be DDR. Figure 5 Taking the example of a computer system including two coprocessors, namely coprocessor 1 and coprocessor 2. Taking the processing unit of the coprocessor as an NPU and the management unit as a CPU as an example for illustration. Specifically, coprocessor 1 includes CPU1, NPU1, and DDR1. Coprocessor 2 includes CPU2, NPU2, and DDR2. The main processor and the coprocessor can be connected by a fast bus that ensures cache coherence. It can be understood that: Host CPU, CPU1, CPU2, NPU1, and NPU2 are connected by a fast bus to form a cache coherence system. DDR1, DDR2, and DDR form a shared memory system.

[0125] See Figure 6 As shown in the figure, based on the above Figure 5 As shown in the figure, it is a schematic flowchart of the data transmission method provided by the present application for the computer system. The tasks to be processed on the main processor are divided into subtask 1 and subtask 2 and stored in the DDR of the main processor.

[0126] S601a, the Host CPU sends the subtask 1 to the coprocessor 1. That is, subtask 1 is migrated from the Host CPU of the main processor from DDR to DDR1.

[0127] S601b, the Host CPU sends the subtask 2 to the coprocessor 2. That is, subtask 2 is migrated from the Host CPU of the main processor from DDR to DDR2.

[0128] Steps S601a and S601b can be implemented in parallel.

[0129] In S602a, after the CPU1 in coprocessor 1 receives subtask 1, it sends subtask 1 to NPU1.

[0130] In S603a, NPU1 starts the process of executing subtask 1 and stores the generated data in the shared memory.

[0131] During the execution of subtask 1 by coprocessor 1, intermediate data is generated, such as data 1 and data 2. Data 1 is stored in the storage space corresponding to storage address 1, and data 2 is stored in the storage space corresponding to storage address 2. Data 1 is required during the running of subtask 2.

[0132] In S602b, after the CPU2 in coprocessor 2 receives subtask 2, it sends subtask 2 to NPU2.

[0133] For example, coprocessor 2 sends subtask 2 to NPU2. Specifically, CPU2 can store subtask 2 in DDR2 and then notify NPU2 of the storage address. Thus, NPU1 reads and executes subtask 2 according to the storage address notified by CPU2.

[0134] In S603b, NPU2 starts the process of executing subtask 2 and stores the generated data in the shared memory.

[0135] During the execution of subtask 2 by coprocessor 2, intermediate data is also generated, such as data 3 and data 4. Data 3 is stored in the storage space corresponding to storage address 3, and data 4 is stored in the storage space corresponding to storage address 4. This data 3 is required during the running of subtask 1.

[0136] In S604a, when NPU1 needs to obtain data 3 generated by NPU2 during the execution of subtask 2 during the execution of subtask 1, it sends a first acquisition request to the CPU1. The first acquisition request is used to request to obtain data 3 generated by coprocessor 2 during the execution of subtask 2.

[0137] In S605a, the CPU1 determines the storage address 3 of the storage space for storing the data 3 according to the mapping relationship and sends the storage address 3 to the NPU1.

[0138] In S606a, the NPU1 reads the data 3 from the storage space corresponding to the storage address 3. Then it continues to execute subtask 1 according to data 3.

[0139] In S604b, when NPU2 needs to obtain data 1 generated by NPU1 during the execution of subtask 1 during the execution of subtask 2, it sends a second acquisition request to the CPU2. The second acquisition request is used to request to obtain data 1 generated by coprocessor 1 during the execution of subtask 1.

[0140] S605b. The CPU 2 determines the storage address 1 of the storage space for storing the data 1 according to the mapping relationship, and sends the storage address 1 to the NPU 2.

[0141] S606b. The NPU 2 reads the data 1 from the storage space corresponding to the storage address 1, and then continues to execute the subtask according to the data 1.

[0142] Regarding the establishment of the mapping relationship, as described above, it will not be elaborated here.

[0143] In a possible implementation, the NPU starts to execute the subtask, which can be triggered by the main processor or by the processing unit in the coprocessor. Taking NPU1 as an example.

[0144] In one example, the Host CPU sends subtask 1 to CPU1; when CPU1 receives subtask 1, it starts to execute subtask 1 immediately.

[0145] In another example, CPU1 receives subtask 1 and the start condition of subtask 1 sent by the main processor, and sends subtask 1 to NPU1. When it is determined that the start condition of subtask 1 is met, NPU1 is triggered to execute subtask 1. After receiving subtask 1, NPU1 starts to execute subtask 1 under the trigger of CPU1.

[0146] See Figure 7As shown in the figure, a task schedule engine (TSE) service can be set in the main processor. The TSE service can run on the main processor in software form, can be deployed on the main processor in the form of a hardware circuit, or can be deployed on the main processor in a combination of software and hardware. TSEs are deployed on all coprocessors. The TSE can run on the CPU of the coprocessor in software form, can be deployed on the coprocessor in the form of a hardware circuit, or can run on the coprocessor in a combination of software and hardware. Taking the main processor as the host CPU as an example, when the TSE service runs on the host CPU in software form, the TSE service can be a service process running on the Host CPU, used to manage the TSE on the coprocessor. The TSE on the coprocessor is responsible for the decision-making, configuration, and startup of data copying, and determines the communication method of NPU1 (or NPU2). For example, when the main processor is the host CPU and the TSE service runs on the host CPU in a combination of software and hardware, the TSE service can be a firmware running in the main processor. Firmware is generally stored in the electrically erasable programmable read-only memory (EEPROM) or flash chip in the device. The program of the firmware generally cannot be directly read or modified by the user. Another example is that the TSE can run on the coprocessor in software form, and the TSE can be a thread or process. Also, when the TSE runs on the coprocessor in a combination of software and hardware, the TSE can be a firmware running on the coprocessor. In the embodiments of this application, the deployment forms of the TSE service and the TSE are not limited.

[0147] For convenience of subsequent description, the TSE of coprocessor 1 is referred to as TSE1, and the TSE of coprocessor 2 is referred to as TSE2.

[0148] When the computer system starts or initializes, the operating system on the main processor is started. The main processor starts the TSE service through the operating system, and then the TSE service starts the TSEs on each coprocessor of the computer system through the PCIe switch chip. The TSE service configures a globally unique device ID for each coprocessor in the computer system and sends the device ID to the TSE of each coprocessor. For example, the device ID of coprocessor 1 is ID1, and the device ID of coprocessor 2 is ID2.

[0149] When the TSE service receives a task to be processed triggered by a user, it divides the task to be processed into two subtasks, namely Task A and Task B. Task A is assigned to NPU1, and Task B is assigned to NPU2.

[0150] In a possible implementation, taking the main processor to create an address information table as an example, the process of NPU1 and NPU2 executing subtasks and obtaining data is described.

[0151] The TSE service pre-creates the mapping relationship between virtual addresses and physical addresses, and creates an address information table based on the device ID of each coprocessor, the identifier of the subtask to be processed, and the already created mapping relationship between virtual addresses and physical addresses. After creating the address information table, in one way, the TSE service can store the created address information table in a shared memory system. And notify the TSE of each coprocessor of the storage address storing the address information table. In another way, the TSE service can send the created address information table to the TSE of each coprocessor respectively.

[0152] After the TSE service divides the task to be processed into Task A and Task B, it sends Task A to TSE1 of coprocessor 1 and sends Task B to TSE2 of coprocessor 2. After TSE1 finishes receiving Task A, it sends a trigger instruction for triggering NPU1 to execute Task A to NPU1. Specifically, the virtual address corresponding to Task A (for example, V1) can be carried in the trigger instruction and sent to NPU1. NPU1 can query the physical address of the data and execution instruction of Task A to be executed from TSE1 according to V1. TSE1 queries the physical address of the data and execution instruction of Task A, such as P1, according to the address translation table managed by TSE1 and sends it to NPU1. Thus, NPU1 reads the data and execution instruction of Task A from the storage space corresponding to P1 and thus executes Task A. Similarly, NPU1 reads the data and execution instruction of Task B and thus executes Task B.

[0153] When NPU1 is executing Task A, during each round of generating intermediate data while executing Task A, it queries TSE1 for the storage address of the intermediate data. TSE1 queries the address information table to obtain the storage address of the intermediate data, such as P1, and sends P1 to NPU1. Thus, NPU1 stores the intermediate data in the storage space corresponding to P1. Similarly, when NPU2 is executing Task B, during each round of generating intermediate data while executing Task B, it queries TSE2 for the storage address of the intermediate data. TSE2 queries the address information table to obtain the storage address of the intermediate data, such as P4, and sends P4 to NPU2. Thus, NPU2 stores the intermediate data in the storage space corresponding to P4.

[0154] If, during the process of NPU2 continuing to execute Task B, it needs to use the intermediate data generated by NPU1 when executing Task A, NPU2 queries the address information table through TSE2 to determine Address 1 for storing Address Translation Table 1. Then TSE2 determines the physical address for storing the intermediate data generated by NPU1 when executing Task A from Address Translation Table 1 stored in the first memory space corresponding to Address 1. After that, TSE2 reads the intermediate data generated by NPU1 when executing Task A from this physical address.

[0155] In another possible implementation, taking the co - processor to allocate storage space and maintain the address information table during the execution of subtasks as an example, the process of NPU1 and NPU2 executing subtasks and obtaining data is described. For example, the address information table is in the form shown in Table 4 and Table 5.

[0156] The TSE service allocates a storage space for each co - processor respectively, for each co - processor to store its corresponding address translation table (Table 5). The TSE service can create Table 4 and then notify the storage address of the created Table 4 to each co - processor. Thus, each co - processor can determine the storage address of the address translation table of other co - processors.

[0157] For example, as shown in Table 4, the TSE service allocates the first memory space in the shared memory to Coprocessor 1 and the second memory space to Coprocessor 2. The storage address of the first memory space is Address 1, and the storage address of the second memory space is Address 2. The TSE service notifies Address 1 and Address 2 to TSE1 of Coprocessor 1 and TSE2 of Coprocessor 2. Thus, the TSEs on Coprocessor 1 and Coprocessor 2 respectively create their own address translation tables and store them in the memory space corresponding to the address allocated by the TSE service.

[0158] TSE1 on coprocessor 1 establishes address translation table 1 in the first memory space corresponding to address 1. TSE of coprocessor 2 establishes address translation table 2 in the second memory space corresponding to address 2.

[0159] The TSE service sends Task A and Task B to the TSE of coprocessor 1 and the TSE of coprocessor 2 respectively. Thus, TSE1 of coprocessor 1 triggers NPU1 to execute Task A, and TSE2 of coprocessor 2 triggers NPU2 to execute Task B. During the execution of Task A by NPU1, the intermediate data generated in each round of executing Task A is stored in DDR1. Similarly, during the execution of Task B by NPU2, the intermediate data generated by executing Task B is stored in DDR2. When NPU2 continues to execute Task B and needs to use the intermediate data generated by NPU1 executing Task A, NPU2 queries the address information table through TSE2 to determine the address 1 for storing address translation table 1. Then, TSE2 determines the physical address for storing the intermediate data generated by NPU1 executing Task A from address translation table 1 stored in the first memory space corresponding to address 1. After that, TSE2 reads the intermediate data generated by NPU1 executing Task A from this physical address.

[0160] Exemplarily, the TSE service first divides the total task into each sub-task and distributes it to the coprocessor side. The sub-task data is also distributed to the coprocessor side. The TSE on the coprocessor side performs initial settings. After allocating storage space in the memory for the data generated during the execution of the sub-task by the NPU, the NPU in each coprocessor starts to execute its respective sub-task. After each round of iterative calculation, the generated data is used as the input data for the next round of iteration. For example, TSE1 allocates storage space in DDR1 for the data generated during the execution of the sub-task by NPU1. Similarly, TSE2 allocates storage space in DDR2 for the data generated during the execution of the sub-task by NPU2. After NPU1 stores the intermediate data generated in each round and notifies the physical address of the stored data to TSE1 for saving in address translation table 1, TSE1 can save the physical address of the intermediate data generated by NPU1 and the virtual address corresponding to the physical address in address translation table 1 after receiving the notification. In addition, after the address translation table 1 managed by TSE1 is updated, TSE1 can notify the TSE service that the address translation table 1 managed by TSE1 has been updated, and the message that the address translation table 1 managed by TSE1 has been updated is notified to the TSE in each coprocessor through the TSE service.

[0161] It should be noted that a virtual address range is configured on TSE1. When TSE1 receives a notification of the physical address where the NPU1 stores intermediate data, it can select a virtual address according to the virtual address range, and store the selected virtual address and the physical address correspondingly in address translation table 1.

[0162] Furthermore, TSE1 notifies TSE2 of the virtual address corresponding to the physical address of this intermediate data. During the process of NPU2 continuing to execute Task B, it needs to use the intermediate data generated by NPU1 when executing Task A. NPU2 notifies TSE2 that it needs to use the intermediate data generated by NPU1 when executing Task A. TSE queries the address information table to determine address 1 for storing address translation table 1. Then, TSE2 determines the physical address for storing the intermediate data generated by NPU1 when executing Task A from address translation table 1 stored in the first memory space corresponding to address 1 according to the virtual address sent by TSE1. After that, TSE2 reads the intermediate data generated by NPU1 when executing Task A from this physical address.

[0163] As an example, take the case where a tensorflow application (app) runs on a host CPU and needs to calculate a two-layer neural network. TensorFlow is an open-source software library for numerical computing that uses data flow graphs. The second-layer neural network needs to apply the calculation result of the first-layer neural network during calculation. See Figure 8 As shown, when the tensorflow app runs on the host CPU and the host CPU detects the start of the tensorflow app, the TSE service in the host CPU is triggered. The TSE service divides the calculation task of the tensorflow app into two calculation tasks: (1) one is sent to NPU1 for calculation through TSE1 on coprocessor 1, and (2) the other is sent to NPU2 for calculation through TSE2 on coprocessor 2. It should be noted that the calculation task of each layer of the neural network is divided into two parts. Thus, after the two NPUs parallelly execute the calculation of the first-layer neural network, they then parallelly execute the calculation of the second-layer neural network.

[0164] First, the TSE service assigns device IDs to coprocessor 1 and coprocessor 2 and sends them to TSE1 and TSE2. TSE1 and TSE2 respectively generate address translation tables. The specific generation method is as described above and will not be elaborated here.

[0165] Secondly, both NPU1 and NPU2 start the calculation of the first-layer neural network.

[0166] During the process of the first-layer neural network in NPU1's computing subtask 1, it will query the address translation table 1 through the local TSE1 to find the physical addresses of the data and instructions of the first-layer neural network. This physical address is dynamically requested and established during the NPU's computing process or pre-constructed and allocated by the CPU in advance. Subsequently, the calculation of the first-layer neural network in subtask 1 begins. When NPU1 completes the calculation of the first-layer neural network in subtask 1, it saves the calculation result of the first-layer neural network in DDR1. NPU1 notifies TSE1 of the physical address 1 of the calculation result of the first-layer neural network in subtask 1. TSE1 assigns a virtual address 1 to the calculation result of the first-layer neural network in this subtask 1 and stores the assigned virtual address 1 and the physical address in the address translation table 1 in correspondence. TSE1 notifies TSE2 of the virtual address 1 of the calculation result of the first-layer neural network in subtask 1.

[0167] Similarly, during the process of the first-layer neural network in NPU2's computing subtask 2, it will query the address translation table 2 through the local TSE2 to find the physical addresses of the data and instructions of the first-layer neural network, and then start the calculation of the first-layer neural network in subtask 2. When NPU2 completes the calculation of the first-layer neural network in subtask 2, it saves the calculation result of the first-layer neural network in DDR2. NPU2 notifies TSE2 of the physical address 2 of the calculation result of the first-layer neural network in subtask 2. TSE2 assigns a virtual address 2 to the calculation result of the first-layer neural network in this subtask 2 and stores the assigned virtual address 2 and the physical address 2 in the address translation table 2 in correspondence. TSE2 notifies TSE1 of the virtual address 2 of the calculation result of the first-layer neural network in subtask 2.

[0168] Finally, NPU1 and NPU2 start calculating the second-layer neural network.

[0169] (3) The NPU1 and NPU2 obtain the sub-task data generated by each other through the TSE. Both the NPU1 and NPU2 require the calculation results of the first-layer neural network for Sub-task 1 and Sub-task 2. The NPU1 starts to calculate the second-layer neural network. The NPU1 queries the address information table through the TSE1 to determine Address 2 for storing Address Translation Table 2. Then, based on the virtual address 2 sent by the TSE2, the TSE1 determines the physical address 2 from the Address Translation Table 2 stored in the second memory space corresponding to Address 2. After that, the TSE1 reads the calculation result of the first-layer neural network in Sub-task 2 from this physical address 2. Similarly, the NPU2 can read the calculation result of the first-layer neural network in Sub-task 1 through the TSE2. Further, the NPU1 performs the calculation of the second-layer neural network for Sub-task 1 based on the calculation results of the first-layer neural networks of Sub-task 1 and Sub-task 2, and the NPU2 performs the calculation of the second-layer neural network for Sub-task 2 based on the calculation results of the first-layer neural networks of Sub-task 1 and Sub-task 2.

[0170] The solution provided in this application implements a High-bandwidth Memory (HM) Link connection management technology. By adding a TSE (Task Scheduling Engine) to the coprocessor and the main processor respectively, for distinction, the one on the main processor is called TSE service. This engine can accurately and relatively quickly identify all devices connected to the HM Link, globally manage the storage addresses of all task data, quickly set the destination and source addresses for data copying according to the service interface, and achieve the ability to perform high-speed data transmission between devices bypassing the PCIe. Through the HM-Link technology, data interaction between coprocessors does not need to obtain and exchange data through the PCIe, that is, zero-copy of data can be achieved, and the coprocessor can directly read the shared memory data.

[0171] The technical solution of this application can be applied to the data flow graph processing process of machine learning and deep learning platforms, and may also be applied to other distributed computing platforms with data exchange requirements. For example, graph computing platforms, streaming computing platforms, etc.

[0172] Those skilled in the art should understand that the embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0173] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single item (s) or plural items (s). For example, at least one (item) of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple. In addition, it should be understood that in the present application, "first", "second", or "third", etc. are only used for distinguishing expressions and have no meaning of limiting the order; in addition, the first object and the second object may be combined or refer to the same object in some cases; furthermore, since there is no limit on the order, there can be a second or a third without a first.

[0174] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for realizing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0175] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing devices to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device realizes the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, so that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable devices provide steps for realizing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0177] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the scope of the embodiments of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these changes and modifications therein.

Claims

1. A data transmission method, characterized in that The method is applied to a first coprocessor and includes: During the execution of a first subtask, determining a first storage address for storing first data according to a mapping relationship, where the mapping relationship includes the association relationship between the identifier of a second coprocessor, the identifier of a second subtask executed by the second coprocessor, and the first storage address, and the first data is the data generated by the second coprocessor during the execution of the second subtask; Reading the first data from a first storage space corresponding to the first storage address; Wherein, the first storage space is located in the shared memory of the first coprocessor and the second coprocessor; the first coprocessor or the second coprocessor is an acceleration device for acceleration.

2. The method according to claim 1, wherein The first coprocessor includes a task scheduling engine, and the task scheduling engine stores program code; The determining the first storage address for storing the first data according to the mapping relationship, where the mapping relationship includes the identifier of the second coprocessor, includes: The first coprocessor runs the program code stored in the task scheduling engine to execute determining the first storage address of the first storage space for storing the first data according to the mapping relationship.

3. The method according to claim 1 or 2, characterized in that, It further includes: Generating second data during the execution of the first subtask, and determining a second storage address according to the mapping relationship; Wherein, the mapping relationship further includes the association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor, and the second storage address; the second storage address corresponds to a second storage space, and the second storage space is used to store the second data, and the second storage space is located in the shared memory; Storing the second data in the second storage space.

4. The method according to claim 1 or 2, characterized in that, It further includes: Generating second data during the execution of the first subtask, determining a second storage space in the shared memory, and storing the second data in the second storage space, where the second storage space corresponds to a second storage address; Adding the association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor, and the second storage address to the mapping relationship.

5. The method according to claim 1 or 2, characterized in that The method further includes: Receiving the first subtask sent by the main processor and starting to execute the first subtask.

6. The method according to claim 1 or 2, characterized in that, The method further includes: Receiving the first subtask sent by the main processor and the start condition of the first subtask; When the start condition of the first subtask is reached, starting to execute the first subtask.

7. The method according to claim 1 or 2, characterized in that, The mapping relationship is stored in the shared memory.

8. A first coprocessor, characterized in that, It includes a first processor and a second processor; The first processor is configured to send a first acquisition request to the second processor during the execution of the first subtask, and the first acquisition request is used to request to acquire the first data generated by the second coprocessor during the execution of the second subtask; The second processor is configured to determine the first storage address of the first storage space for storing the first data according to the mapping relationship and send the first storage address to the first processor; Among them, the mapping relationship includes the association relationship between the identifier of the second coprocessor, the identifier of the second subtask executed by the second coprocessor, and the first storage address; The first processor is configured to read the first data from the first storage space corresponding to the first storage address; Among them, the first storage space is located in the shared memory of the first coprocessor and the second coprocessor; the first processor is an accelerator; the second processor is a general-purpose processor.

9. The first coprocessor according to claim 8, characterized in that, The first coprocessor further includes a task scheduling engine, and the task scheduling engine stores program code; The second processor is specifically configured to run the program code stored in the task scheduling engine to execute the first storage address of the first storage space for storing the first data determined according to the mapping relationship, and send the first storage address to the first processor.

10. The first coprocessor according to claim 8 or 9, characterized in that: When the first processor is further configured to store the second data generated during the execution of the first subtask, the first processor sends a second acquisition request to the second processor, and the second acquisition request is used to request the storage space for storing the second data; The second processor is further configured to, after receiving the second acquisition request, determine a second storage address according to the mapping relationship, and send the second storage address to the first processor; Among them, the mapping relationship further includes the association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor, and the second storage address; the second storage address corresponds to a second storage space, and the second storage space is used to store the second data generated during the execution of the first subtask by the first coprocessor; The first processor is further configured to store the second data in the second storage space corresponding to the second storage address.

11. The first coprocessor according to claim 8 or 9, characterized in that: When the first processor is further configured to store the second data generated during the execution of the first subtask, the first processor sends a third acquisition request to the second processor, and the third acquisition request is used to request the storage space for storing the second data; The second processor is further configured to, after receiving the third acquisition request, allocate a second storage space for the first processor, the second storage space corresponds to a second storage address, notify the first processor of the second storage address, and add the association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor, and the second storage address to the mapping relationship; The first processor is further configured to store the second data in the second storage space corresponding to the second storage address.

12. The first coprocessor according to claim 8 or 9, characterized in that: The second processor is further configured to receive the first subtask sent by the main processor and send the first subtask to the first processor; The first processor is further configured to, when receiving the first subtask, start executing the first subtask.

13. The first coprocessor according to claim 8 or 9, characterized in that: The second processor is further configured to receive the first subtask issued by the main processor and the start condition of the first subtask, and send the first subtask to the first processor, and when it is determined that the start condition of the first subtask is met, trigger the first processor to execute the first subtask; The first processor is further configured to receive the first subtask and start to execute the first subtask under the trigger of the second processor.

14. A computer system, characterized in that, It includes a first coprocessor and a second coprocessor; The first coprocessor is configured to execute a first subtask; The second coprocessor is configured to execute a second subtask; the first coprocessor or the second coprocessor is an acceleration device for acceleration; The first coprocessor is further configured to determine a first storage address for storing the first data according to the mapping relationship during the execution of the first subtask, and read the first data from the first storage space corresponding to the first storage address; The first data is data generated by the second coprocessor during the execution of the second subtask; The mapping relationship includes the association relationship between the identifier of the second coprocessor, the identifier of the second subtask executed by the second coprocessor and the first storage address; Wherein, the first storage space is located in the shared memory of the first coprocessor and the second coprocessor.

15. The system according to claim 14, wherein The first coprocessor further includes a task scheduling engine, and the task scheduling engine stores program codes; The first coprocessor is specifically configured to run the program codes stored in the task scheduling engine to execute the first storage address of the first storage space for storing the first data determined according to the mapping relationship.

16. The system according to claim 14 or 15, characterized in that The first coprocessor is further configured to: Generate second data during the execution of the first subtask, and determine a second storage address according to the mapping relationship; Wherein, the mapping relationship further includes the association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor and the second storage address; the second storage address corresponds to a second storage space, and the second storage space is used to store the second data generated during the execution of the first subtask by the first coprocessor; Store the second data in the second storage space corresponding to the second storage address.

17. The system according to claim 14 or 15, characterized in that The first coprocessor is further configured to generate second data during the execution of the first subtask, and store the second data in a second storage space, and the second storage space corresponds to a second storage address; Add the association relationship between the identifier of the first coprocessor, the identifier of the first subtask executed by the first coprocessor and the second storage address to the mapping relationship.

18. The system according to claim 14 or 15, characterized in that, The first coprocessor and the second coprocessor are deployed in one chip.

19. The system according to claim 14 or 15, characterized in that, It further includes a main processor: The main processor is configured to issue the first subtask to the first coprocessor and issue the second subtask to the second coprocessor.

20. The system according to claim 19, wherein The main processor and the first coprocessor and the second coprocessor are deployed in different chips.

21. The system according to claim 20, wherein, The main processor is further configured to send the start condition of the first subtask to the first coprocessor, and send the start condition of the second subtask to the second coprocessor; The first coprocessor is specifically configured to start executing the first subtask when it determines that the start condition of the first subtask is met; The second coprocessor is specifically configured to start executing the second subtask when it determines that the start condition of the second subtask is met.

22. The system according to claim 21, wherein The first coprocessor includes a first processor and a second processor; The second processor is configured to receive the first subtask from the main processor and the start condition of the first subtask, and send the first subtask to the first processor; The second processor is further configured to trigger the first processor to execute the first subtask when it determines that the start condition of the first subtask is met; The first processor is configured to execute the first subtask under the trigger of the second processor.

23. A chip, characterized in that, The chip includes one or more first coprocessors as described in any one of claims 8-13.

Images