First graphic processor, remote communication method, chip, server and device

Abstract

The application discloses a first graphics processor, a remote communication method, a chip, a server and equipment, and belongs to the chip field.The first graphics processor comprises a plurality of processors, a data loading engine and a global storage unit, the data loading engine comprises a remote communication channel;the remote communication channel is used for receiving a remote communication instruction sent by the plurality of processors;and the remote communication channel is used for reading target data from the global storage unit in response to the remote communication instruction and sending the target data to a second graphics processor.The application sets the remote communication channel on the data loading engine, reads the target data by the remote communication channel when the plurality of processors execute a remote communication task, and sends the target data to the second graphics processor, without temporarily storing and forwarding the target data by the storage resource in the plurality of processors, so that the internal resource of the plurality of processors can be inclined to a calculation task, and the calculation task and the remote communication task can be concurrently executed on the same plurality of processors.

Description

Technical Field

[0001] This application relates to the field of chips, and in particular to a first graphics processor, a remote communication method, a chip, a server, and a device. Background Technology

[0002] A graphics processing unit (GPU) includes a multi-processor (MP) and a global memory unit. The multi-processor has an internal local memory unit and a vector general-purpose register (VGPR).

[0003] When the graphics processor performs a computational task, the multiprocessor will move the data required for the computation from the global memory to the local memory and store the computation results in the internal vector registers. When the graphics processor performs a remote communication task (the task of transferring data from one graphics processor to another), it will also occupy the multiprocessor's internal vector registers and local memory.

[0004] Therefore, due to resource limitations of vector registers and local storage units, computational tasks and remote communication tasks are usually not concurrent on the same multiprocessor. Summary of the Invention

[0005] This application provides a first graphics processor, a remote communication method, a chip, a server, and a device. This application sets up a remote communication path on the data loading engine. When the multiprocessor executes a remote communication task, the target data will be read by the remote communication path and sent to the second graphics processor without being temporarily stored and forwarded by the internal storage resources of the multiprocessor. Thus, the internal resources of the multiprocessor can be allocated to the computing task, and the computing task and the remote communication task can run concurrently on the same multiprocessor.

[0006] According to one aspect of this application, a first graphics processor is provided, the first graphics processor including a multiprocessor, a data loading engine and a global storage unit, the data loading engine including a remote communication path;

[0007] A remote communication channel is used to receive remote communication commands sent by multiple processors;

[0008] A remote communication channel is used to read target data from the global storage unit and send the target data to the second graphics processor in response to a remote communication command.

[0009] According to another aspect of this application, a remote communication method is provided, the method being applied to a first graphics processor, the first graphics processor including a multiprocessor, a data loading engine and a global storage unit, the data loading engine including a remote communication path, the method comprising the following steps.

[0010] The remote communication channel receives remote communication commands sent by multiple processors;

[0011] In response to a remote communication command, the remote communication channel reads the target data from the global storage unit and sends the target data to the second graphics processor.

[0012] According to one aspect of this application, a chip is provided, the chip including the first graphics processor described above.

[0013] According to one aspect of this application, a server is provided, the server including the first graphics processor described above.

[0014] According to one aspect of this application, an electronic device is provided, the electronic device including the first graphics processor described above.

[0015] The beneficial effects of the technical solutions provided in this application include at least the following:

[0016] In the hardware architecture of the graphics processor provided in this application, the data loading engine is located on the first graphics processor and acts as a coprocessor for the multiprocessor to assist it in performing remote communication tasks. When the multiprocessor performs a remote communication task, the target data is read from the remote communication path on the data loading engine and sent directly to the second graphics processor, without having to send the target data to the multiprocessor itself. Instead, the target data is temporarily stored and forwarded by the multiprocessor's internal vector registers and local storage units. Therefore, the remote communication scheme provided in this application does not occupy the internal resources of the multiprocessor, allowing the multiprocessor's internal resources to be allocated to computational tasks. Consequently, the method provided in this application allows computational tasks and remote communication tasks to run concurrently on the same multiprocessor, thus fully utilizing the multiprocessor's computing power. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of a remote communication method provided in one embodiment of this application.

[0019] Figure 2 This is a schematic diagram of a remote communication method provided in another embodiment of this application.

[0020] Figure 3 This is a schematic diagram of a cross-card transmission method for the i-th sub-data provided in one embodiment of this application.

[0021] Figure 4 This is a schematic diagram of a cross-card transmission method for the i-th sub-data provided in another embodiment of this application.

[0022] Figure 5 This is a schematic diagram of the structure of a data loading engine provided in one embodiment of this application.

[0023] Figure 6 This is a schematic diagram of the structure of a first graphics processor provided in one embodiment of this application.

[0024] Figure 7 This is a schematic diagram of a remote communication method provided in another embodiment of this application.

[0025] Figure 8 This is a schematic diagram of the structure of a first graphics processor provided in another embodiment of this application.

[0026] Figure 9 This is a flowchart of a remote communication method provided in one embodiment of this application.

[0027] Figure 10 This is a flowchart of a remote communication method provided in another embodiment of this application.

[0028] Figure 11 This is a structural block diagram of an electronic device provided in one embodiment of this application.

[0029] Figure 12 This is a schematic diagram of the structure of a server provided in one embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0031] First, some terms used in the embodiments of this application will be introduced:

[0032] Computational tasks: The multiprocessors on the graphics processing unit (GPU) will perform computational tasks. These tasks refer to the computational operations performed by the multiprocessors, such as matrix multiplication. When the multiprocessors perform computational tasks, they need to move the data required for the computation from the GPU's global memory to the multiprocessor's internal local memory, and they also need to move the computation results from the vector registers to the global memory.

[0033] Remote communication tasks: The multiprocessors on the graphics processing unit (GPU) will perform remote communication tasks, which refer to the transfer of data from one GPU (the local GPU where the multiprocessor resides) to another GPU. When performing remote communication tasks, the multiprocessors need to transfer data from the global memory of one GPU to the global memory of another GPU.

[0034] Multiprocessor: A graphics processing unit (GPU) comprises multiple multiprocessors, each with multiple cores. When the GPU executes a task, the compute front-end controller (CFH) initiates several thread blocks and assigns each thread block to a multiprocessor for execution. Each multiprocessor executes the task instructions carried by one thread block. Within a multiprocessor, the assigned thread blocks are further divided into multiple thread bundles. A multiprocessor executes the instructions carried by multiple threads within each thread bundle, and for all threads within the same thread bundle, the multiprocessor executes the same instructions within the same cycle.

[0035] Figure 1 A schematic diagram of a remote communication method provided in an exemplary embodiment of this application is shown. Figure 1 A first graphics processor 100 and a second graphics processor 101 are shown. Optionally, the first graphics processor 100 and the second graphics processor 101 are graphics processors within the same computer device or graphics processors within different computer devices. Optionally, the computer device includes any one of a terminal device and a server.

[0036] like Figure 1 As shown, the first graphics processor 100 includes a multiprocessor 102, a data loading engine 103, and a global storage unit 104. The data loading engine 103 includes a remote communication path 1031. Optionally, the multiprocessor 102 can be any one of the multiprocessors within the first graphics processor 100. The remote communication path 1031 is a hardware path within the data loading engine 103 used for remotely transferring data. The remote communication path 1031 is used to perform remote copy operations, which include transferring data from one graphics processor (GPU card) to another graphics processor (another GPU card). Optionally, the global storage unit 104 is a storage unit within the first graphics processor 100 to support data read / write operations of multiple multiprocessors.

[0037] Indicative Figure 1In the illustrated remote communication method, the multiprocessor 102 sends a remote communication command to the remote communication path 1031, which instructs the remote communication path 1031 to perform a remote copy operation. In response to the remote communication command, the remote communication path 1031 reads the target data from the global storage unit 104 and sends the target data to the second graphics processor 101. Optionally, in response to the remote communication command, the remote communication path 1031 reads the target data from the global storage unit 104 and directly sends the target data to the second graphics processor 101 without temporary storage. That is, when the remote communication path 1031 forwards the target data, it does not occupy storage resources and does not store the target data.

[0038] Optionally, the data loading engine 103 corresponds to one multiprocessor 102, and the remote communication path 1031 in the data loading engine 103 is used to assist one multiprocessor 102 in performing remote communication tasks; or, the data loading engine 103 corresponds to multiple multiprocessors 102, and the remote communication path 1031 in the data loading engine 103 is used to assist multiple multiprocessors 102 in performing remote communication tasks.

[0039] It is understood that in the hardware structure of the graphics processor provided in this application, the data loading engine 103 is located inside the first graphics processor 100 and acts as a coprocessor for the multiprocessor 102 to assist the multiprocessor 102 in performing remote communication tasks. When the multiprocessor 102 performs a remote communication task, the remote communication path 1031 on the data loading engine 103 reads the target data and sends the target data directly to the second graphics processor 101, without sending the target data to the multiprocessor 102. The target data is then temporarily stored and forwarded by the vector registers and local storage units inside the multiprocessor 102. Therefore, the remote communication method provided in this application does not occupy the internal resources of the multiprocessor 102. The internal resources of the multiprocessor 102 can be allocated to computing tasks. Consequently, the method provided in this application allows computing tasks and remote communication tasks to run concurrently on the same multiprocessor 102, making full use of the computing power resources of the multiprocessor 102.

[0040] In one embodiment, the device type of the terminal device includes at least one of the following: smartphone, smartwatch, in-vehicle terminal, wearable device, smart TV, tablet computer, e-book reader, MP3 player, MP4 player, laptop computer, and desktop computer.

[0041] Optionally, the server can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0042] In one embodiment, the multiprocessor 102 within the first graphics processor 100, in addition to sending remote communication instructions to the remote communication path 1031, also sends the target data's data read address, data storage address, and target size to the remote communication path 1031. The data read address is the address used by the remote communication path 1031 when reading the target data from the global storage unit 104. The data storage address is the address used by the remote communication path 1031 when storing the target data in the second graphics processor 101.

[0043] Optionally, the data storage address is the address of the global memory unit inside the second graphics processor 101.

[0044] Target size refers to the size of the target data, for example, the target data to be transferred is 32 bits.

[0045] Optionally, the remote communication instruction is a macro instruction. Only the data read address, data storage address, and target size need to be provided for the remote communication channel 1031 to execute the remote communication instruction.

[0046] Schematic, the remote communication path 1031 receives a remote communication command, the data read address, the data storage address, and the target size sent by the multiprocessor 102. In response to the remote communication command, the remote communication path 1031 reads the target data from the global storage unit 104 according to the data read address and the target size; then, according to the data storage address, it sends the target data to the second graphics processor 101.

[0047] In one embodiment, the remote communication channel 1031 reads the target data from the global storage unit 104 at once and sends the target data directly to the second graphics processor 101.

[0048] In another embodiment, the remote communication path 1031 reads the target data from the global storage unit 104 in multiple steps, sending a portion of the target data to the second graphics processor 101 each time. In some cases, the target data to be transported by the remote communication task is large. If the target data is transported all at once, the remote communication task will occupy a large portion of the bandwidth of the global storage unit 104, thus affecting the execution of the computing task, which will be unable to read / write data from the global storage unit 104. Therefore, this embodiment uses a multiple small-volume transport method, which can allocate the bandwidth of the global storage unit 104 to the computing task as much as possible, realizing the parallelism of the remote communication task and the computing task.

[0049] Indicative, such as Figure 2 As shown, in response to a remote communication command, the remote communication path 1031 generates multiple sub-loading requests. The multiple sub-loading requests are generated one by one by the remote communication path 1031. The remote communication path 1031 sends each generated sub-loading request to the global storage unit 104, reads the sub-data corresponding to each sub-loading request from the global storage unit 104. The sub-data is a part of the target data. Then, the remote communication path 1031 sends each sub-data to the second graphics processor 101.

[0050] Optionally, the remote communication path 1031 generates sub-loading requests one by one. Each time a sub-loading request is generated, it is sent to the global storage unit 104. After reading the corresponding sub-data, the sub-data is sent to the second graphics processor 101. That is, each time the remote communication path 1031 sends a sub-loading request, upon receiving the corresponding sub-data, it immediately sends the sub-data to the second graphics processor 101. At any given time, there are several sub-loading requests between the remote communication path 1031 and the global storage unit 104.

[0051] In one embodiment, the multiprocessor 102 includes a scalar register, which is the most basic storage unit in the multiprocessor 102, used to temporarily store and manipulate single data values. The remote communication path 1031 reads the data read address, data storage address, and target size from the scalar register. In this embodiment, the data read address, data storage address, and target size required are scalar information, which can be stored using only the scalar register. There is no need for the multiprocessor 102 to allocate vector registers; the multiprocessor 102 only needs to allocate scalar registers. This allows for the parallelization of remote communication tasks and computation tasks (computation tasks require the use of vector registers) without increasing resources within the multiprocessor 102.

[0052] In one embodiment, the data loading engine 103 includes an instruction cache unit. The instruction cache unit caches remote communication instructions, data read addresses, data storage addresses, and target sizes. The instruction cache unit caches these information so that each time a sub-load request, sub-read address, sub-size, and sub-store address are generated, the corresponding instructions or data can be quickly retrieved for processing. For the data read address, data storage address, and target size, there is no need to read them again from the scalar registers of the multiprocessor 102.

[0053] Optionally, the instruction cache unit also stores fixed data types, such as F32, S32, U32, F16, BF16, S16, U16, S8, U8, E5M2, E4M3, and E2M1, with the latter three being specific formats for FP8 and FP4. Optionally, the instruction cache unit also stores the identifier (ID) of the asynchronous barrier module, used to indicate which asynchronous barrier module is enabled. Information about asynchronous barrier modules will be described below.

[0054] The following section will describe the i-th sub-load request among multiple sub-load requests.

[0055] The i-th sub-load request is any one of multiple sub-load requests, where i is a positive integer greater than one.

[0056] like Figure 3 As shown, the remote communication channel 1031 will send the i-th sub-load request to the global storage unit 104 to obtain the i-th sub-data returned by the global storage unit 104. The i-th sub-data is the data requested to be loaded by the i-th sub-load request. The remote communication channel 1031 will send the i-th sub-data to the second graphics processor 101.

[0057] In one embodiment, in response to a remote communication command, the remote communication path 1031 generates an i-th sub-load request among multiple sub-load requests, and generates an i-th sub-read address corresponding to the i-th sub-load request based on the data read address, wherein the i-th sub-read address is the starting address on the global storage unit 104 accessed by the i-th sub-load request; and generates an i-th sub-size corresponding to the i-th sub-load data based on the target size, wherein the i-th sub-size is the size of the i-th sub-data. The remote communication path 1031 reads the i-th sub-data from the global storage unit 104 according to the i-th sub-load request, the i-th sub-read address, and the i-th sub-size.

[0058] Schematic illustration: The remote communication channel 1031 starts the state machine, using the data read address as the base address and (i-1) times the fixed size as the offset address to obtain the i-th sub-read address. For example, if the original data read address is x and the fixed size is 128 bytes, then the first sub-read address is x, the second sub-read address is x+128, the third sub-read address is x+256, and so on. The fixed size represents the size of the data loaded from the global storage unit 104 each time.

[0059] To illustrate, when the target size is an integer multiple of the fixed size, the sub-size corresponding to each sub-load request is the fixed size. However, if the target size is not an integer multiple of the fixed size, when the i-th sub-load request is the last sub-load request, the i-th sub-size will not be the fixed size; when the i-th sub-load request is not the last sub-load request, the i-th sub-size is the fixed size. In this case, if the i-th sub-load request is the last sub-load request, the i-th sub-size will be smaller than the fixed size. For example, if the fixed size is 128 bytes, the i-th sub-size could be 1, 2, 3, ..., 127 bytes, etc.

[0060] In one embodiment, the size of the data loaded from the global storage unit 104 by each of the multiple sub-loading requests is fixed; in conjunction with reference to... Figure 4 , Figure 4 This illustrates a schematic process by which the remote communication path 1031 reads the i-th sub-data from the global storage unit 104. For example... Figure 4 As shown, the remote communication path 1031 sends the i-th sub-load request and the i-th sub-read address to the global storage unit 104 to obtain the i-th loaded data returned by the global storage unit 104; the remote communication path 1031 aligns the i-th loaded data according to the i-th sub-size to obtain the i-th sub-data.

[0061] Alignment refers to checking the size of the i-th loaded data. If the size of the i-th loaded data does not match the i-th sub-size, the i-th loaded data is split according to the i-th sub-size. If the size of the i-th loaded data matches the i-th sub-size, the i-th loaded data is determined as the i-th sub-data.

[0062] For example, each time the remote communication channel 1031 reads the loaded data from the global storage unit 104, it is of a fixed size. If the target size is not an integer multiple of the fixed size, and if the i-th sub-load request is not the last sub-load request, the size of the i-th sub-load request will be a fixed size (for example, 128 bytes). In this case, the remote communication channel 1031 will align the i-th loaded data according to the fixed size, determine that the size of the i-th loaded data is also a fixed size, and determine the i-th loaded data as the i-th sub-data. If the i-th sub-load request is the last sub-load request, the size of the i-th sub-load request will be smaller than the fixed size (for example, the size of the i-th sub-load request is 1, 2, 3 bytes, etc.). In this case, the remote communication channel 1031 will align the i-th loaded data according to the i-th sub-size, and segment the data that meets the i-th sub-size from the i-th loaded data as the i-th sub-data. At this time, only a portion of the data in the i-th loaded data is needed to form the complete target data with the data loaded by other sub-load requests.

[0063] Optionally, after the remote communication path 1031 generates the i-th sub-size corresponding to the i-th sub-load request, it will store the i-th sub-size in the matching table unit; after obtaining the i-th loading data, the remote communication path 1031 matches the i-th sub-size from the matching table unit according to the request identifier of the i-th sub-load request.

[0064] The details of matching table cells will be discussed below.

[0065] By matching the table cell to record the i-th sub-size, it is easier to use the i-th sub-size to align the i-th loaded data read from the global storage unit 104. For cases where the target size is not an integer multiple of the fixed size, it can avoid sending irrelevant data to the second graphics processor 101.

[0066] In one embodiment, after the remote communication path 1031 reads the i-th sub-data, it determines the i-th sub-storage address corresponding to the i-th sub-data based on the data storage address and the i-th storage offset address, and sends the i-th sub-data to the i-th sub-storage address on the second graphics processor 101.

[0067] Optionally, after generating the i-th sub-loading request, the remote communication path 1031 stores the i-th storage offset address corresponding to the i-th sub-loading request in the matching table unit; after reading the i-th sub-data, the remote communication path 1031 matches the i-th storage offset address from the matching table unit according to the request identifier of the i-th sub-loading request, and then generates the i-th sub-storage address based on the data storage address and the i-th storage offset address, and sends the i-th sub-data to the i-th sub-storage address on the second graphics processor 101.

[0068] Schematic illustration: Before issuing the i-th sub-load request, the remote communication channel 1031 generates the i-th storage offset address. For example, the remote communication channel 1031 uses a fixed size of (i-1) times as the i-th storage offset address. After reading the i-th sub-data, the remote communication channel 1031 uses the data storage address as the base address and combines it with the i-th storage offset address to obtain the i-th sub-storage address. For example, if the original data storage address is y and the fixed size is 128 bytes, then the first sub-storage address is y, the second sub-storage address is y+128, the third sub-storage address is y+256, and so on.

[0069] By recording the offset address of the i-th sub-data in the matching table unit, it is easy to calculate the i-th sub-data address using the offset address, thus accurately determining the storage location of the i-th sub-data on the second graphics processor 101. Recording in the matching table unit also facilitates the recording and management of relevant information for each sub-loading request, ensuring that even with a large number of sub-loading requests, each request can be executed in an orderly and accurate manner.

[0070] Matching table cells

[0071] In one embodiment, such as Figure 5 As shown, the data loading engine 103 also includes a matching table unit 1032, which is a hardware unit in the data loading engine 103 used for recording and querying data.

[0072] As described above, after the remote communication channel 1031 issues the i-th sub-loading request, it will record the i-th sub-size and / or the i-th storage offset address corresponding to the i-th sub-loading request in the matching table unit 1032. Optionally, the matching table unit 1032 supports querying by the request identifier of the sub-loading request. Optionally, the matching table unit 1032 also records the identifier (ID) of the remote communication instruction to which the sub-loading request belongs.

[0073] The matching table cell 1032 is shown in the following table as an example:

[0074]

[0075] In the matching table unit 1032 shown above, sub-load requests 1, 2, and 3 are sub-load requests generated based on remote communication instruction 1, and sub-load request 4 is a sub-load request generated based on remote communication instruction 2. Furthermore, sub-load requests 1, 2, and 4 are not the last sub-load requests among their respective sub-load requests, and the sub-size of sub-load requests 1, 2, and 4 is 128 bytes. Sub-load request 3 is the last sub-load request among its respective sub-load requests, and its sub-size is 8 bytes. Additionally, in the matching table unit 1032, the storage offset addresses corresponding to sub-load requests 1 to 4 are 128, 256, 384, and 0, respectively.

[0076] By matching table unit 1032, the sub-size and storage offset address of each sub-load request can be recorded, which makes it easier to use the sub-size to align the loaded data read from global storage unit 104 and use the storage offset address to determine the sub-storage address corresponding to each sub-data.

[0077] In this embodiment, the matching table unit 1032 includes multiple entries, each corresponding one-to-one with a sub-loading request (i.e., each entry corresponds to one sub-loading request), and the multiple sub-loading requests are generated based on the same remote communication command. Optionally, for the i-th entry among the multiple entries, the i-th entry is used to record the i-th sub-size and / or the i-th storage offset address corresponding to the i-th sub-loading request. Schematic, as shown in the matching table unit above, the i-th entry is used to record the i-th sub-size, the i-th storage offset address, and the remote communication command identifier to which the i-th sub-loading request belongs.

[0078] In one embodiment, such as Figure 5 As shown, the data loading engine 103 also includes a first register 1033. The first register 1033 is used to configure the number of available entries in the matching table unit 1032. Schematic, the matching table unit 1032 can hold a total of 256 entries, and when the value stored in the first register is configured to be 128, the remote communication path 1031 can only use 128 of those entries.

[0079] In this embodiment, by setting a first register 1033 in the data loading engine 103, the first register 1033 is used to configure the number of available entries in the matching table unit 1032. This can avoid channel congestion caused by upstream over-issuance (for example, the remote communication channel 1031 generating too many sub-loading requests in a short period of time), and ensure the orderly and stable operation of the remote communication task.

[0080] In one embodiment, after receiving the i-th acknowledgment signal sent by the second graphics processor 101, the remote communication path 1031 clears the i-th entry; the i-th acknowledgment signal is used to indicate that the second graphics processor 101 has received the i-th sub-data. In this embodiment, the remote communication path 1031 releases the corresponding entry in the matching table unit 1032 each time it receives an acknowledgment signal, which can avoid the matching table unit 1032 storing outdated information and reserve sufficient entry resources for subsequent sub-loading requests.

[0081] Asynchronous barrier module

[0082] In one embodiment, such as Figure 6 As shown, the first graphics processor 100 also includes an asynchronous barrier module 105, which is a hardware unit for implementing the asynchronous barrier function. Optionally, the asynchronous barrier module 105 is used to notify the multiprocessor 102 that the remote communication task has been completed when all sub-remote communication tasks (tasks that send sub-data to the second graphics processor 101) have been completed.

[0083] In this embodiment, the remote communication path 1031 generates sub-loading requests one by one and sends the sub-data corresponding to each sub-loading request. That is, the remote communication path 1031 transfers each sub-data asynchronously. The asynchronous barrier module 105 is used to record the size of each transferred sub-data. When the accumulated size of the transferred data reaches the target size of the target data, it notifies the multiprocessor 102 that the remote communication task of all data has been completed, thus forming a closed loop of the remote communication task.

[0084] Indicatively, after receiving the i-th confirmation signal sent by the second graphics processor 101, the remote communication channel 1031 adds the value corresponding to the i-th sub-size to the cumulative size value stored in the asynchronous barrier module 105. The i-th confirmation signal is used to indicate that the second graphics processor 101 has received the i-th sub-data.

[0085] When the cumulative size value reaches the value corresponding to the target size, the asynchronous barrier module 105 notifies the multiprocessor 102 that the remote communication task of all data has been completed.

[0086] Optionally, the asynchronous barrier module 105 will also notify the multiprocessor 102 of the completion of the stage progress of the remote communication task when the cumulative size value reaches the stage value. For example, if the stage value is 1 / 3 of the target size, the asynchronous barrier module 105 will notify the multiprocessor 102 that 1 / 3 of the remote communication task has been completed.

[0087] In one embodiment, the first graphics processor 100 has multiple asynchronous barrier modules. The remote communication path 1031 adds the value corresponding to the i-th sub-size to the cumulative size value stored in the corresponding asynchronous barrier module 105 according to the identifier (ID) of the asynchronous barrier module. Optionally, the identifier of the asynchronous barrier module 105 is stored in the instruction cache unit mentioned above. Optionally, the identifier of the asynchronous barrier module 105 is sent by the multiprocessor 102 to the data loading engine 103 and stored in the instruction cache unit.

[0088] In this embodiment, the asynchronous barrier module 105 can continuously record the size of the data sent. When the size of the data sent reaches the target size, the corresponding thread bundle on the multiprocessor 102 is woken up and the thread bundle is informed that the remote communication task of all data has been completed. The asynchronous barrier module 105 ensures that the multiprocessor 102 can know that the remote communication task has been completed when multiple small-volume data transfers are used. The multiprocessor 102 confirms that the remote communication task has been completed based on the received notification.

[0089] Remote atomic operation instructions

[0090] In one embodiment, in addition to responding to remote communication instructions to send target data on global storage unit 104 to second graphics processor 101, remote communication channel 1031 can also respond to remote atomic operation instructions to obtain data to be operated from global storage unit 104, perform atomic operations on the data to be operated, and send the atomic operation results to second graphics processor 101.

[0091] Indicative, such as Figure 7 As shown, the remote communication path 1031 receives a remote atomic operation instruction sent by the multiprocessor 102; in response to the remote atomic operation instruction, the remote communication path 1031 reads the data to be operated from the global storage unit 104, performs atomic operations on the data to be operated and the data on the second graphics processor 101, and sends the atomic operation result to the second graphics processor 101.

[0092] Atomic operations include any one of the following: NOT, SUM, XOR, AND, and OR operations.

[0093] Optionally, the remote communication path 1031 performs atomic operations on the data to be operated on and the data at the storage address of the second graphics processor 101, and sends the atomic operation results to the storage address.

[0094] Alternatively, it can be understood that the remote communication channel 1031 directly uses the data to be operated on to perform atomic operations on the data at the storage address of the second graphics processor 101.

[0095] In one embodiment, the remote communication path 1031 receives the address to be read, the address to be stored, and the size of the data to be operated sent by the multiprocessor 102; in response to a remote atomic operation instruction, the remote communication path 1031 reads the data to be operated from the global storage unit 104 according to the address to be read and the size of the data to be operated; performs an atomic operation on the data to be operated and the data at the address to be stored in the second graphics processor 101, and sends the result of the atomic operation to the address to be stored.

[0096] Optionally, the remote atomic operation instruction is a macro instruction. Only the address to be read, the address to be stored, and the size of the data to be operated are required, and the remote communication channel 1031 can execute the remote atomic operation instruction.

[0097] In one embodiment, the remote communication path 1031, in response to a remote atomic operation instruction, generates a j-th sub-load request among multiple sub-load requests; generates a j-th sub-read address corresponding to the j-th sub-load request based on the address to be read; and generates a j-th sub-size corresponding to the j-th sub-load request based on the size of the data to be operated on; the remote communication path 1031 reads the j-th sub-data from the global storage unit 104 according to the j-th sub-load request, the j-th sub-read address, and the j-th sub-size; determines the j-th sub-storage address corresponding to the j-th sub-data based on the address to be stored; performs atomic operations on the j-th sub-data and the data at the j-th sub-storage address of the second graphics processor 101, and sends the atomic operation result to the j-th sub-storage address;

[0098] The data to be operated on is obtained by loading multiple sub-loading requests. These sub-loading requests are generated one by one, and the j-th sub-loading request is any one of the multiple sub-loading requests.

[0099] It is understandable that remote atomic operation instructions require a large amount of data to be operated on. If the data to be operated on is read all at once, the remote atomic operation instructions will occupy a large portion of the bandwidth of the global storage unit 104, which will affect the execution of the computing task, preventing the computing task from reading / writing data from the global storage unit 104. Therefore, this embodiment adopts a method of multiple small-volume reads, which can allocate the bandwidth of the global storage unit 104 to the computing task as much as possible.

[0100] In one embodiment, the size of the data loaded by each of the multiple sub-loading requests is fixed. The remote communication path 1031 sends the j-th sub-loading request and the j-th sub-read address to the global storage unit 104, obtains the j-th loaded data returned by the global storage unit 104, and aligns the j-th loaded data according to the j-th sub-size to obtain the j-th sub-data.

[0101] Alignment refers to checking the size of the j-th loaded data. If the size of the j-th loaded data does not match the j-th sub-size, the j-th loaded data is split according to the j-th sub-size. If the size of the j-th loaded data matches the j-th sub-size, the j-th loaded data is determined as the j-th sub-data.

[0102] In one embodiment, the data loading engine 103 further includes a matching table unit 1032;

[0103] After generating the j-th sub-size corresponding to the j-th sub-load request, the remote communication channel 1031 stores the j-th sub-size in the matching table unit 1032; after obtaining the j-th loading data, it matches the j-th sub-size from the matching table unit 1032 according to the request identifier of the j-th sub-load request.

[0104] In one embodiment, the remote communication path 1031 determines the j-th sub-storage address corresponding to the j-th sub-data based on the address to be stored and the j-th storage offset address.

[0105] Optionally, after generating the j-th sub-loading request, the remote communication path 1031 stores the j-th storage offset address corresponding to the j-th sub-loading request in the matching table unit; after reading the j-th sub-data, the remote communication path 1031 matches the j-th storage offset address from the matching table unit according to the request identifier of the j-th sub-loading request, and then generates the j-th sub-storage address based on the data storage address and the j-th storage offset address, performs atomic operations on the j-th sub-data and the data at the j-th sub-storage address of the second graphics processor 101, and sends the atomic operation result to the j-th sub-storage address.

[0106] Schematic illustration: Before issuing the j-th sub-load request, the remote communication channel 1031 generates the j-th storage offset address. For example, the remote communication channel 1031 uses a fixed size of (j-1) times as the j-th storage offset address. After reading the j-th sub-data, the remote communication channel 1031 uses the data storage address as the base address and combines it with the j-th storage offset address to obtain the j-th sub-storage address. For example, if the original data storage address is y and the fixed size is 128 bytes, then the first sub-storage address is y, the second sub-storage address is y+128, the third sub-storage address is y+256, and so on.

[0107] Recording the j-th sub-size in the matching table facilitates subsequent alignment of the j-th loaded data read from global storage unit 104 with this sub-size. For cases where the size of the data to be operated on is not an integer multiple of a fixed size, it avoids using irrelevant data to perform atomic operations. Recording the j-th sub-storage offset address in the matching table facilitates subsequent calculation of the j-th sub-storage address, accurately determining the storage location on the second graphics processor 101 of the data that performs atomic operations with the j-th sub-data. Recording in the matching table also facilitates the recording and management of relevant information for each sub-loading request, ensuring that even with a large number of sub-loading requests, each request can be executed in an orderly and accurate manner.

[0108] In one embodiment, the multiprocessor 102 includes a scalar register, which is the most basic storage unit in the multiprocessor 102, used for temporarily storing and manipulating single data values. The remote communication path 1031 reads the address to be read, the address to be stored, and the size of the data to be manipulated from the scalar register. In this embodiment, the address to be read, the address to be stored, and the size of the data to be manipulated are scalar information, which can be stored using only the scalar register. There is no need for the multiprocessor 102 to allocate vector registers; the multiprocessor 102 only needs to allocate scalar registers. This allows resources to be allocated to computing tasks (which require vector registers) without increasing resources within the multiprocessor 102.

[0109] In one embodiment, the data loading engine 103 further includes an instruction cache unit; the instruction cache unit is used to cache remote atomic operation instructions, addresses to be read, addresses to be stored, and the size of data to be operated on. The instruction cache unit caches remote atomic operation instructions, addresses to be read, addresses to be stored, and the size of data to be operated on, so that each time a sub-load request, sub-read address, sub-size, and sub-store address are generated, the corresponding instructions or data can be quickly read from them for processing. For addresses to be read, addresses to be stored, and the size of data to be operated on, there is no need to read them from the scalar registers of the multiprocessor 102.

[0110] In one embodiment, the first graphics processor 100 further includes an asynchronous barrier module 105;

[0111] After receiving the j-th confirmation signal sent by the second graphics processor 101, the remote communication channel 1031 adds the value corresponding to the j-th sub-size to the cumulative size value stored in the asynchronous barrier module 105. The j-th confirmation signal is used to indicate that the second graphics processor 101 has received the atomic operation result corresponding to the j-th sub-data.

[0112] The asynchronous barrier module 105 notifies the multiprocessor that the remote atomic operation task of all data has been completed when the cumulative size value reaches the value corresponding to the size of the data to be operated.

[0113] In one embodiment, the data loading engine 103 further includes a matching table unit 1032; the matching table unit 1032 includes a j-th entry, the j-th entry is used to record the j-th sub-size and / or the j-th storage offset address corresponding to the j-th sub-loading request, and the j-th storage offset address is used to determine the j-th sub-storage address corresponding to the j-th sub-data.

[0114] After receiving the j-th confirmation signal sent by the second graphics processor 101, the remote communication channel 1031 clears the j-th entry; the j-th confirmation signal is used to indicate that the second graphics processor has received the atomic operation result corresponding to the j-th sub-data.

[0115] In this embodiment, each time the remote communication channel 1031 receives an acknowledgment signal, it releases the corresponding entry in the matching table unit 1032, which can prevent the matching table unit 1032 from storing outdated information and reserve sufficient entry resources for subsequent sub-loading requests.

[0116] In one embodiment, the data loading engine 103 further includes a matching table unit 1032 and a first register 1033; the matching table unit 1032 includes multiple entries, each of which corresponds to a multiple sub-loading request; the j-th entry in the multiple entries is used to record the j-th sub-size and / or the j-th storage offset address corresponding to the j-th sub-loading request; the j-th storage offset address is used to determine the j-th sub-storage address corresponding to the j-th sub-data.

[0117] The first register 1033 is configured to match the number of available entries in the matching table unit 1032.

[0118] In this embodiment, by setting a first register 1033 in the data loading engine 103, the first register 1033 is used to configure the number of available entries in the matching table unit 1032. This can avoid channel congestion caused by upstream over-issuance (for example, the remote communication channel 1031 generating too many sub-loading requests in a short period of time), and ensure the orderly and stable operation of the remote communication task.

[0119] It should be noted that for details on how the remote communication channel 1031 responds to remote atomic operation instructions, please refer to the relevant content on how the remote communication channel 1031 responds to remote communication instructions mentioned above. Detailed explanations will not be provided here. Similarities between the two schemes can be referenced from each other.

[0120] The above describes the detailed process of remote communication tasks assisted by remote communication path 1031. At this time, when the multiprocessor 102 executes the remote communication task, it will not occupy the resources of the internal vector register and local storage unit. The resources of the vector register and local storage unit can be allocated to the computing task. Therefore, the remote communication task and the computing task can run concurrently on the same multiprocessor 102.

[0121] Next, we will consider the bandwidth allocation issue of the global storage unit 104 when remote communication tasks and computing tasks are executed concurrently on the same multiprocessor 102. In related technologies, in some rare cases, remote communication tasks and computing tasks have been able to run concurrently on the same multiprocessor. However, the remote communication tasks and computing tasks executing on the same multiprocessor will compete for the bandwidth of the global storage unit 104. In this case, the remote communication tasks may cause the computing tasks to be unable to obtain bandwidth, thereby affecting the data loading and storage of the computing tasks.

[0122] The technical solution provided in this application will be described below.

[0123] In one embodiment, such as Figure 8 As shown, the data loading engine 103 includes a remote communication path 1031 and a local data path 1034. The local data path 1034 is used to read and / or write data in the global storage unit 104 when the multiprocessor 102 performs computing tasks.

[0124] Optionally, the local data path 1034 includes a data loading path (LD pipe) and a data storage path (ST pipe). The data loading path is used to load the data required for the computing task from the global storage unit 104 into the local storage unit within the multiprocessor 102. The data storage path is used to store the computing results from the local storage unit into the global storage unit 104.

[0125] Priority

[0126] In one embodiment, the remote communication path 1031 and the local data path 1034 share the hardware port on the data loading engine 103 that is connected to the global storage unit 104; the remote communication path 1031 will preempt the hardware port according to the preemption priority configured in the second register.

[0127] The hardware port is the port on the data loading engine 103 used for data transmission.

[0128] Schematic, the data loading engine 103 includes a second register for storing the preemption priority of the remote communication path 1031 and / or the local data path 1034. For example, if the preemption priority of the remote communication path 1031 in the second register is lower than that of the local data path 1034, then when the multiprocessor 102 executes remote communication tasks and computing tasks in parallel, the remote communication path 1031 will not preempt or will preempt less of the hardware ports shared with the local data path 1034.

[0129] In another embodiment, the remote communication path 1031 and the local data path 1034 share the hardware port on the data loading engine 103 that is connected to the global storage unit 104; the remote communication path 1031 preempts the hardware port according to the preemption priority specified by the priority instruction.

[0130] Schematic, preemption priority can also be indicated by priority instructions. The multiprocessor 102 sends priority instructions to the data loading engine 103, and the data loading engine 103 adjusts the preemption priority of the internal remote communication path 1031 and / or local data path 1034 according to the priority instructions. For example, if the priority instruction indicates that the preemption priority of the remote communication path 1031 is lower than that of the local data path 1034, then when the multiprocessor 102 executes remote communication tasks and computing tasks in parallel, the remote communication path 1031 will not preempt or will preempt less of the hardware ports shared with the local data path 1034.

[0131] In the above embodiments, by setting the preemption priority, the hardware port can be configured to be used more by the local data path 1034. At this time, the remote communication task will not occupy or will occupy very little bandwidth resources of the computing task, thus minimizing the performance interference between the computing task and the remote communication task.

[0132] Preemption time interval

[0133] In one embodiment, the remote communication path 1031 and the local data path 1034 share a hardware port on the data loading engine 103 that is connected to the global storage unit 104; the remote communication path 1031 preempts the hardware port according to the preemption time interval configured in the third register.

[0134] Schematic, the data loading engine 103 includes a third register for storing the preemption intervals of the remote communication path 1031 and / or the local data path 1034. For example, if the preemption interval of the remote communication path 1031 stored in the third register is 10 clock cycles, then when the multiprocessor 102 executes remote communication tasks and computing tasks in parallel, the remote communication path 1031 preempts the hardware port shared with the local data path 1034 once every 10 clock cycles.

[0135] In one embodiment, the remote communication path 1031 and the local data path 1034 share the hardware port on the data loading engine 103 that is connected to the global storage unit 104; the remote communication path 1031 preempts the hardware port according to the preemption time interval specified by the time interval instruction.

[0136] Schematic, the preemption interval can also be indicated by a time interval instruction. The multiprocessor 102 sends a time interval instruction to the data loading engine 103, and the data loading engine 103 adjusts the preemption interval of the internal remote communication path 1031 and / or local data path 1034 according to the time interval instruction. For example, if the time interval instruction indicates that the preemption interval of the remote communication path 1031 is 10 clock cycles, then when the multiprocessor 102 executes remote communication tasks and computing tasks in parallel, the remote communication path 1031 preempts the hardware port shared with the local data path 1034 once every 10 clock cycles.

[0137] In the above embodiments, by setting the preemption time interval, the hardware port can be configured to be used more by the local data path 1034. At this time, the remote communication task will not occupy or will occupy very little bandwidth resources of the computing task, thus minimizing the performance interference between the computing task and the remote communication task.

[0138] Monitoring local data pathways

[0139] Since both the remote communication path 1031 and the local data path 1034 are located in the data loading engine 103, the remote communication path 1031 can have the function of monitoring the local data path 1034.

[0140] In one embodiment, the remote communication path 1031 monitors the local data path 1034 and determines the frequency of sending sub-load requests to the global storage unit 104 based on the number of read and write requests on the local data path 1034. The sub-load requests are used to read a portion of the target data from the global storage unit.

[0141] Read / write requests on the local data path 1034 refer to requests sent by the local data path 1034 to the global storage unit 104 for reading / writing data. As described above, the remote communication path 1031 will send sub-load requests to the global storage unit 104 multiple times, each sub-load request being used to read corresponding sub-data from the global storage unit. In this embodiment, the remote communication path 1031 can determine the frequency of sending sub-load requests to the global storage unit 104 according to the number of read / write requests on the local data path 1034. When there are fewer read / write requests on the local data path 1034, the remote communication path 1031 can send more sub-load requests in a short period of time; when there are more read / write requests on the local data path 1034, the remote communication path 1031 can send fewer or no sub-load requests in a short period of time.

[0142] Therefore, by monitoring the number of read and write requests on the local data path 1034, the remote communication path 1031 can adaptively adjust the sending frequency of sub-loading requests, so as not to preempt or preempt less bandwidth of the global storage unit 104. At this time, the remote communication task will not occupy or occupy very little bandwidth resources of the computing task, thus minimizing the performance interference between the computing task and the remote communication task.

[0143] Flow control

[0144] Typically, the cross-card bandwidth requirement between two graphics processors is relatively small. Even if the remote communication path 1031 reads a large amount of data from the global storage unit 104 at once, it still takes a considerable amount of time to send it to the other graphics processor. Therefore, the time interval for the remote communication path 1031 to send sub-load requests can be set based on the cross-card bandwidth requirement between the two graphics processors. A larger time interval for sending sub-load requests, determined based on the smaller cross-card bandwidth requirement, means that the remote communication path 1031 will only preempt the bandwidth of the global storage unit 104 once every few weeks. This minimizes the bandwidth resources consumed by the computing task and reduces performance interference between the computing task and the remote communication task.

[0145] In one embodiment, the remote communication path 1031 determines the time interval for sending a sub-load request to the global storage unit 104 according to the cross-card bandwidth requirements between the first graphics processor 100 and the second graphics processor 101. The sub-load request is used to read a portion of the target data from the global storage unit 104.

[0146] For example, the cross-card bandwidth requirement between the first graphics processor 100 and the second graphics processor 101 is 300GB / s. The first graphics processor 100 has 64 multiprocessors performing remote communication tasks. The frequency of the first graphics processor 100 is 1.5GHz. Then, each multiprocessor only needs 3.125B / clk bandwidth to fully meet the cross-card bandwidth requirement when sending data to the second graphics processor 101.

[0147] Assuming that a sub-load request loads sub-data of 128B (usually the size of a cache line), the remote communication path 1031 only needs to send a sub-load request to the global storage unit 104 every 40 cycles. After reading the corresponding 128B sub-data, it can forward it to the second graphics processor 101.

[0148] As can be seen, according to the cross-card bandwidth requirements between the first graphics processor 100 and the second graphics processor 101, only one sub-load request needs to be sent every 40 cycles, and the remaining 39 cycles can be used by the local data path 1034. At this time, the remote communication task will occupy very little bandwidth resources of the computing task, minimizing the performance interference between the computing task and the remote communication task.

[0149] Figure 9 A flowchart of a remote communication method provided in an embodiment of this application is shown. Taking the application of this method to a first graphics processor as an example, the first graphics processor includes a multiprocessor, a data loading engine, and a global storage unit. The data loading engine includes a remote communication path. The method includes:

[0150] Step 920: The remote communication channel receives remote communication instructions sent by the multiprocessor;

[0151] Optionally, the multiprocessor can be any one of the multiprocessors within the first graphics processor.

[0152] Optionally, the data loading engine corresponds to one multiprocessor, and the remote communication path within the data loading engine assists the multiprocessor in performing remote communication tasks; alternatively, the data loading engine corresponds to multiple multiprocessors, and the remote communication path within the data loading engine assists the multiple multiprocessors in performing remote communication tasks. Optionally, the remote communication path is a hardware path within the data loading engine used for remotely moving data. The remote communication path is used to perform remote copy operations, which include moving data from one graphics processor (GPU card) to another graphics processor (another GPU card).

[0153] Optionally, the global storage unit is a storage unit inside the first graphics processor to support data read and write operations of multiple processors.

[0154] For example, the remote communication path receives a remote communication instruction sent by the multiprocessor, which instructs the remote communication path to perform a remote copy operation.

[0155] In step 940, in response to the remote communication command, the remote communication channel reads the target data from the global storage unit and sends the target data to the second graphics processor.

[0156] In one embodiment, in response to a remote communication command, the remote communication path reads target data from the global storage unit and sends the target data to the second graphics processor. Optionally, in response to a remote communication command, the remote communication path reads target data from the global storage unit and sends the target data directly to the second graphics processor without pausing. That is, when the remote communication path forwards target data, it does not occupy storage resources and does not store the target data.

[0157] In summary, in the hardware architecture of the graphics processor provided in this application, the data loading engine is located inside the first graphics processor and acts as a coprocessor for the multiprocessor to assist it in performing remote communication tasks. When the multiprocessor performs a remote communication task, the target data is read from the remote communication path on the data loading engine and sent directly to the second graphics processor, without needing to send the target data to the multiprocessor itself. Instead, the target data is temporarily stored and forwarded by the multiprocessor's internal vector registers and local storage units. Therefore, the remote communication method provided in this application does not occupy the internal resources of the multiprocessor, allowing these resources to be allocated to computational tasks. Consequently, the method provided in this application allows computational tasks and remote communication tasks to run concurrently on the same multiprocessor, fully utilizing the multiprocessor's computing power.

[0158] based on Figure 9 In the optional embodiment shown, step 920 further includes: receiving the data read address, data storage address, and target size of the target data sent by the multiprocessor via a remote communication channel.

[0159] The data read address is the address used by the remote communication path to read target data from the global storage unit. The data storage address is the address used by the remote communication path to store target data in the second graphics processor. Optionally, the data storage address is the address of a global storage unit within the second graphics processor.

[0160] Target size refers to the size of the target data, for example, the target data to be transferred is 32 bits.

[0161] Optionally, the remote communication command is a macro command, which only requires providing the data read address, data storage address and target size, and the remote communication channel can execute the remote communication command.

[0162] Step 940 includes: in response to a remote communication command, the remote communication path reads target data from the global storage unit according to the data read address and data size; and sends the target data to the second graphics processor according to the data storage address.

[0163] Optionally, the multiprocessor includes scalar registers; a remote communication path reads data from the scalar registers, including the read address, data storage address, and target size.

[0164] Optionally, the data loading engine also includes an instruction cache unit; the instruction cache unit caches remote communication instructions, data read addresses, data storage addresses, and target sizes.

[0165] Figure 10 A flowchart of a remote communication method provided in an exemplary embodiment of this application is shown, illustrated by an example of the method being applied to a first graphics processor. The first graphics processor includes a multiprocessor, a data loading engine, and a global storage unit. The data loading engine includes a remote communication path. The method includes the following steps.

[0166] Step 1010: In response to the remote communication command, the remote communication channel generates the i-th sub-loading request among multiple sub-loading requests;

[0167] The target data is obtained by loading multiple sub-loading requests, which are generated one by one. The i-th sub-loading request is any one of the multiple sub-loading requests.

[0168] Step 1020: The remote communication channel generates the i-th sub-read address corresponding to the i-th sub-load request based on the data read address;

[0169] Schematic illustration: The remote communication path initiates a state machine, using the data read address as the base address and a fixed size of (i-1) times as the offset address to obtain the i-th sub-read address. For example, if the original data read address is x and the fixed size is 128 bytes, then the first sub-read address is x, the second sub-read address is x+128, the third sub-read address is x+256, and so on. The fixed size represents the size of the data loaded from global storage unit 104 each time.

[0170] Step 1030: The remote communication channel generates the i-th sub-size corresponding to the i-th sub-loading request based on the target size;

[0171] To illustrate, when the target size is an integer multiple of the fixed size, the sub-size corresponding to each sub-load request is the fixed size. However, if the target size is not an integer multiple of the fixed size, when the i-th sub-load request is the last sub-load request, the i-th sub-size will not be the fixed size; when the i-th sub-load request is not the last sub-load request, the i-th sub-size is the fixed size. In this case, if the i-th sub-load request is the last sub-load request, the i-th sub-size will be smaller than the fixed size. For example, if the fixed size is 128 bytes, the i-th sub-size could be 1, 2, 3, ..., 127 bytes, etc.

[0172] Step 1040: The remote communication channel reads the i-th sub-data from the global storage unit according to the i-th sub-load request, the i-th sub-read address, and the i-th sub-size;

[0173] In one embodiment, the size of the data loaded by each of the multiple sub-loading requests is a fixed size; a remote communication channel sends the i-th sub-loading request and the i-th sub-read address to the global storage unit, and obtains the i-th loaded data returned by the global storage unit; the i-th loaded data is aligned according to the i-th sub-size to obtain the i-th sub-data.

[0174] Alignment refers to checking the size of the i-th loaded data. If the size of the i-th loaded data does not match the i-th sub-size, the i-th loaded data is split according to the i-th sub-size. If the size of the i-th loaded data matches the i-th sub-size, the i-th loaded data is determined as the i-th sub-data.

[0175] For example, if the remote communication channel reads a fixed-size load data from the global storage unit each time, and the target size is not an integer multiple of the fixed size, then if the i-th sub-load request is not the last sub-load request, the i-th sub-size will be a fixed size (e.g., 128 bytes). In this case, the remote communication channel will align the i-th load data according to the fixed size, ensuring that the size of the i-th load data is also fixed, and thus designate the i-th load data as the i-th sub-data. If the i-th sub-load request is the last sub-load request, the i-th sub-size will be smaller than the fixed size (e.g., the i-th sub-size is 1, 2, or 3 bytes). In this case, the remote communication channel will align the i-th load data according to the i-th sub-size, and segment the data that meets the i-th sub-size from the i-th load data as the i-th sub-data. In this case, only a portion of the data from the i-th load data is needed to form the complete target data with the data loaded by the other sub-load requests.

[0176] Optionally, the data loading engine also includes a matching table unit; a remote communication channel, which stores the i-th sub-size in the matching table unit after generating the i-th sub-size corresponding to the i-th sub-loading request; and matches the i-th sub-size from the matching table unit according to the request identifier of the i-th sub-loading request after obtaining the i-th loading data.

[0177] By matching the table cell to record the i-th sub-size, it is easier to use the i-th sub-size to align the i-th loaded data read from the global storage cell. For cases where the target size is not an integer multiple of the fixed size, it can avoid sending irrelevant data to the second graphics processor.

[0178] Step 1050: The remote communication channel determines the i-th sub-storage address corresponding to the i-th sub-data based on the data storage address;

[0179] In one embodiment, the remote communication path determines the i-th sub-storage address corresponding to the i-th sub-data based on the data storage address and the i-th storage offset address.

[0180] Optionally, the data loading engine also includes a matching table unit; a remote communication path, which stores the i-th storage offset address corresponding to the i-th sub-loading request in the matching table unit after generating the i-th sub-loading request among multiple sub-loading requests; and a remote communication path, which matches the i-th storage offset address from the matching table unit according to the request identifier of the i-th sub-loading request after reading the i-th sub-data.

[0181] To illustrate, before issuing the i-th sub-load request, the remote communication path generates the i-th storage offset address. For example, the remote communication path uses a fixed size of (i-1) times the i-th storage offset address. After reading the i-th sub-data, the remote communication path uses the data storage address as the base address and combines it with the i-th storage offset address to obtain the i-th sub-storage address. For example, if the original data storage address is y and the fixed size is 128 bytes, then the first sub-storage address is y, the second sub-storage address is y+128, the third sub-storage address is y+256, and so on.

[0182] By recording the offset address of the i-th sub-data in the matching table unit, it is easy to calculate the i-th sub-data address using the offset address, thus accurately determining the storage location of the i-th sub-data on the second graphics processor 101. Recording in the matching table unit also facilitates the recording and management of relevant information for each sub-loading request, ensuring that even with a large number of sub-loading requests, each request can be executed in an orderly and accurate manner.

[0183] Step 1060: The remote communication channel sends the i-th sub-data to the second graphics processor according to the i-th sub-storage address.

[0184] The remote communication path stores the i-th sub-data in the i-th sub-memory address of the second graphics processor.

[0185] In summary, in the above embodiments, the remote communication path reads the target data from the global storage unit multiple times, sending a portion of the target data to the second graphics processor each time. In some cases, the target data to be transported by the remote communication task is large. If the target data is transported all at once, the remote communication task will occupy a large portion of the global storage unit's bandwidth, thus affecting the execution of the computing task, which will be unable to read / write data from the global storage unit. Therefore, this embodiment uses a multiple small-volume transport method, which can allocate the bandwidth of the global storage unit as much as possible to the computing task, achieving parallelism between the remote communication task and the computing task.

[0186] In one embodiment, the first graphics processor further includes an asynchronous barrier module. Following step 1060 above, the method further includes: after receiving the i-th confirmation signal sent by the second graphics processor, the remote communication path adds the value corresponding to the i-th sub-size to the cumulative size value stored in the asynchronous barrier module; the i-th confirmation signal indicates that the second graphics processor has received the i-th sub-data.

[0187] When the cumulative size value reaches the value corresponding to the target size, the asynchronous barrier module notifies the multiprocessor that the remote communication task of all data has been completed.

[0188] Optionally, the asynchronous barrier module is a hardware unit used to implement the asynchronous barrier function. Optionally, the asynchronous barrier module is used to notify the multiprocessor that the remote communication task has been completed when all sub-remote communication tasks (tasks that send sub-data to the second graphics processor) have been completed.

[0189] Optionally, the asynchronous barrier module will also notify the multiprocessor of the progress of the remote communication task when the cumulative size value reaches a certain stage. For example, if the stage value is 1 / 3 of the target size, the asynchronous barrier module will notify the multiprocessor that 1 / 3 of the remote communication task has been completed.

[0190] Optionally, multiple asynchronous barrier modules exist on the first graphics processor. The remote communication path adds the value corresponding to the i-th sub-size to the cumulative size value stored in the corresponding asynchronous barrier module according to the identifier (ID) of the asynchronous barrier module. Optionally, the identifier of the asynchronous barrier module 105 is stored in the instruction cache unit mentioned above. Optionally, the identifier of the asynchronous barrier module is sent by the multiprocessor to the data loading engine 103 and stored in the instruction cache unit.

[0191] In summary, in this embodiment, the asynchronous barrier module can continuously record the size of the sent data. When the sent size reaches the target size, the corresponding thread bundle on the multiprocessor is awakened, and the thread bundle is informed that the remote communication task of all data has been completed. The asynchronous barrier module ensures that the multiprocessor can know that the remote communication task has been completed when multiple small-volume data transfers are used. The multiprocessor confirms the completion of the remote communication task based on the received notification.

[0192] In one embodiment, the data loading engine further includes a matching table unit; the matching table unit includes an i-th entry, the i-th entry records the i-th sub-size and / or the i-th storage offset address corresponding to the i-th sub-load request, and the i-th storage offset address determines the i-th sub-storage address corresponding to the i-th sub-data.

[0193] Following step 1060 above, the method further includes: a remote communication path, which, upon receiving the i-th acknowledgment signal sent by the second graphics processor, clears the i-th entry; the i-th acknowledgment signal indicates that the second graphics processor has received the i-th sub-data. In this embodiment, the remote communication path releases the corresponding entry in the matching table unit each time it receives an acknowledgment signal, which can prevent the matching table unit from storing outdated information and reserve sufficient entry resources for subsequent sub-loading requests.

[0194] In one embodiment, the data loading engine further includes a matching table unit and a first register; the matching table unit includes multiple entries, each corresponding one-to-one with a multiple sub-loading request, wherein the i-th entry records the i-th sub-size and / or the i-th storage offset address corresponding to the i-th sub-loading request, and the i-th storage offset address determines the i-th sub-storage address corresponding to the i-th sub-data; the method further includes:

[0195] According to the number of available entries in the matching table unit configured in the first register, the i-th entry in the matching table unit records the i-th sub-size and / or the i-th storage offset address corresponding to the i-th sub-load request.

[0196] In this embodiment, by setting a first register in the data loading engine, which is used to configure the number of available entries in the matching table unit, the path congestion caused by upstream over-issuance (for example, too many sub-loading requests generated in a short period of time in the remote communication path) can be avoided, thus ensuring the orderly and stable operation of the remote communication task.

[0197] In one embodiment, the method further includes: a remote communication channel, and receiving a remote atomic operation instruction sent by a multiprocessor; in response to the remote atomic operation instruction, reading data to be operated from a global storage unit, performing atomic operations on the data to be operated and data on a second graphics processor, and sending the atomic operation result to the second graphics processor;

[0198] Atomic operations include any one of the following: NOT, SUM, XOR, AND, and OR operations.

[0199] In one embodiment, the data loading engine further includes a local data path, which enables the reading and / or writing of data in the global storage unit when a multiprocessor performs a computing task; the remote communication path and the local data path share a hardware port on the data loading engine connected to the global storage unit; the method further includes:

[0200] The remote communication channel preempts the hardware port according to the preemption priority configured in the second register; or, according to the preemption priority specified by the priority instruction, it preempts the hardware port.

[0201] In the above embodiments, by setting the preemption priority, the hardware port can be configured to be used more by the local data path. In this case, the remote communication task will not occupy or will occupy very little bandwidth resources of the computing task, thus minimizing the performance interference between the computing task and the remote communication task.

[0202] In one embodiment, the data loading engine further includes a local data path, which enables the reading and / or writing of data in the global storage unit when a multiprocessor performs a computing task; the remote communication path and the local data path share a hardware port on the data loading engine connected to the global storage unit; the method further includes:

[0203] The remote communication channel preempts the hardware port according to the preemption time interval configured in the third register; or, preempts the hardware port according to the preemption time interval specified by the time interval instruction.

[0204] In the above embodiments, by setting a preemption time interval, the hardware port can be configured to be used more by the local data path. In this case, the remote communication task will not occupy or will occupy very little bandwidth resources of the computing task, thus minimizing the performance interference between the computing task and the remote communication task.

[0205] In one embodiment, the data loading engine further includes a local data path, which enables the reading and / or writing of data in a global storage unit when a multiprocessor performs a computing task; the method further includes:

[0206] The remote communication path monitors the local data path. Based on the number of read and write requests on the local data path, the frequency of sending sub-load requests to the global storage unit is determined. The sub-load requests read a portion of the target data from the global storage unit.

[0207] In the above embodiments, by monitoring the number of read and write requests on the local data path, the remote communication path can adaptively adjust the sending frequency of sub-loading requests, so as not to preempt or preempt less bandwidth of the global storage unit. At this time, the remote communication task will not occupy or occupy very little bandwidth resources of the computing task, thus minimizing the performance interference between the computing task and the remote communication task.

[0208] In one embodiment, the method further includes: a remote communication path, determining the time interval for sending a sub-load request to the global storage unit according to the cross-card bandwidth requirements between the first graphics processor and the second graphics processor, wherein the sub-load request reads a portion of the target data from the global storage unit.

[0209] Typically, the cross-card bandwidth requirement between two graphics processors is relatively small. In this case, even if the remote communication path reads a large amount of data from the global storage unit at once, it still takes a considerable amount of time to send it to the other graphics processor. In the above embodiment, based on the cross-card bandwidth requirement between the two graphics processors, the time interval for the remote communication path to send sub-load requests is set. The time interval for sending sub-load requests, determined based on the smaller cross-card bandwidth requirement, is relatively large. Therefore, the remote communication path will only preempt the global storage unit's bandwidth once every few seconds, and the remote communication task will consume very little bandwidth resources from the computing task, minimizing performance interference between the computing task and the remote communication task.

[0210] Figure 11 A structural block diagram of an electronic device 1100 provided in an exemplary embodiment of this application is shown. Optionally, the electronic device 1100 includes a first graphics processor provided in an embodiment of this application.

[0211] Optionally, the electronic device can be a portable mobile terminal, such as a smartphone, tablet, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop, or desktop computer. The electronic device 1100 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names. Typically, the electronic device 1100 includes a processor 1101 and a memory 1102.

[0212] Processor 1101 may include one or more processing cores, such as a quad-core processor, an eleven-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0213] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices.

[0214] In some embodiments, the electronic device 1100 may optionally include: a peripheral device interface 1103 and at least one peripheral device. Those skilled in the art will understand that... Figure 11 The structure shown does not constitute a limitation on the electronic device 1100, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0215] This application also provides a processor, which includes a first graphics processor as described in the above embodiments.

[0216] This application also provides a chip that includes the first graphics processor described in the above embodiments.

[0217] This application also provides a graphics card that includes a first graphics processor as described in the above embodiments.

[0218] This application also provides a server that includes a first graphics processor as described in the embodiments above.

[0219] Figure 12 The diagram shows a schematic of the structure of a server provided in an exemplary embodiment of this application. The server 1200 includes a plurality of graphics processors 1201, and at least one graphics processor 1201 is a first graphics processor as described in the above embodiment.

[0220] This application also provides a computing cluster, which includes multiple servers, at least one of which includes the first graphics processor described in the above embodiments.

[0221] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0222] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0223] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A first graphics processor, characterized in that, The first graphics processor includes a multiprocessor, a data loading engine, and a global storage unit, wherein the data loading engine includes a remote communication path; The remote communication channel is used to receive remote communication instructions sent by the multiprocessor; The remote communication channel is used to respond to the remote communication command, read target data from the global storage unit, and send the target data directly to the second graphics processor without temporary storage.

2. The first graphics processor according to claim 1, characterized in that, The remote communication channel is used to receive the data read address, data storage address, and target size of the target data sent by the multiprocessor; The remote communication path is used to respond to the remote communication command, read the target data from the global storage unit according to the data read address and the target size, and send the target data to the second graphics processor according to the data storage address.

3. The first graphics processor according to claim 2, characterized in that, The remote communication channel is used to generate the i-th sub-loading request among multiple sub-loading requests in response to the remote communication instruction; And based on the data read address, generate the i-th sub-read address corresponding to the i-th sub-load request; And based on the target size, generate the i-th sub-size corresponding to the i-th sub-load request; The remote communication path is used to read the i-th sub-data from the global storage unit according to the i-th sub-loading request, the i-th sub-read address, and the i-th sub-size; The remote communication path is used to determine the i-th sub-storage address corresponding to the i-th sub-data based on the data storage address; and to send the i-th sub-data to the second graphics processor according to the i-th sub-storage address. The target data is obtained by loading the plurality of sub-loading requests, which are generated one by one. The i-th sub-loading request is any one of the plurality of sub-loading requests, where i is a positive integer greater than one.

4. The first graphics processor according to claim 3, characterized in that, The size of the data loaded by each of the multiple sub-loading requests is a fixed size; The remote communication channel is used to send the i-th sub-load request and the i-th sub-read address to the global storage unit, and to obtain the i-th loaded data returned by the global storage unit. The i-th loaded data is aligned with the i-th sub-size to obtain the i-th sub-data.

5. The first graphics processor according to claim 4, characterized in that, The data loading engine also includes a matching table unit; The remote communication channel is used to store the i-th sub-size in the matching table unit after generating the i-th sub-size corresponding to the i-th sub-loading request; The remote communication channel is used to match the i-th sub-size from the matching table unit according to the request identifier of the i-th sub-loading request after acquiring the i-th loaded data.

6. The first graphics processor according to claim 3, characterized in that, The remote communication path is used to determine the i-th sub-storage address corresponding to the i-th sub-data based on the data storage address and the i-th storage offset address.

7. The first graphics processor according to claim 6, characterized in that, The data loading engine also includes a matching table unit; The remote communication path is used to store the i-th storage offset address corresponding to the i-th sub-loading request in the matching table unit after generating the i-th sub-loading request among the plurality of sub-loading requests; The remote communication path is used to match the i-th storage offset address from the matching table unit according to the request identifier of the i-th sub-load request after reading the i-th sub-data.

8. The first graphics processor according to any one of claims 2 to 7, characterized in that, The data loading engine also includes an instruction cache unit; The instruction cache unit is used to cache the remote communication instruction, the data read address, the data storage address, and the target size.

9. The first graphics processor according to any one of claims 3 to 7, characterized in that, The first graphics processor also includes an asynchronous barrier module; The remote communication channel is used to add the value corresponding to the i-th sub-size to the cumulative size value stored in the asynchronous barrier module after receiving the i-th confirmation signal sent by the second graphics processor. The i-th confirmation signal is used to indicate that the second graphics processor has received the i-th sub-data. The asynchronous barrier module is used to notify the multiprocessor that the remote communication task of all data has been completed when the cumulative size value reaches the value corresponding to the target size.

10. The first graphics processor according to any one of claims 3 to 7, characterized in that, The data loading engine further includes a matching table unit; the matching table unit includes an i-th entry, the i-th entry is used to record at least one of the i-th sub-size and the i-th storage offset address corresponding to the i-th sub-loading request, the i-th storage offset address is used to determine the i-th sub-storage address corresponding to the i-th sub-data; The remote communication channel is used to clear the i-th entry after receiving the i-th confirmation signal sent by the second graphics processor; The i-th confirmation signal is used to indicate that the second graphics processor has received the i-th sub-data.

11. The first graphics processor according to any one of claims 2 to 7, characterized in that, The multiprocessor includes a scalar register; the remote communication path is used to read the data read address, the data storage address, and the target size from the scalar register.

12. The first graphics processor according to any one of claims 3 to 7, characterized in that, The data loading engine further includes a matching table unit and a first register; the matching table unit includes multiple entries, each of which corresponds one-to-one with the multiple sub-loading requests, wherein the i-th entry is used to record at least one of the i-th sub-size and the i-th storage offset address corresponding to the i-th sub-loading request, and the i-th storage offset address is used to determine the i-th sub-storage address corresponding to the i-th sub-data; The first register is configured to be used to determine the number of available entries in the matching table unit.

13. The first graphics processor according to any one of claims 1 to 7, characterized in that, The remote communication channel is also used to receive remote atomic operation instructions sent by the multiprocessor; The remote communication channel is also used to respond to the remote atomic operation instruction, read the data to be operated from the global storage unit, perform an atomic operation on the data to be operated and the data on the second graphics processor, and send the atomic operation result to the second graphics processor; The atomic operations include any one of the following: NOT, SUM, XOR, AND, and OR operations.

14. The first graphics processor according to any one of claims 1 to 7, characterized in that, The data loading engine also includes a local data path, which is used to read and write data in the global storage unit when the multiprocessor performs computing tasks; the remote communication path and the local data path share the hardware port on the data loading engine that is connected to the global storage unit. The remote communication path is used to preempt the hardware port according to the preemption priority configured in the second register; or, according to the preemption priority specified by the priority instruction, to preempt the hardware port.

15. The first graphics processor according to any one of claims 1 to 7, characterized in that, The data loading engine also includes a local data path, which is used to read and write data in the global storage unit when the multiprocessor performs computing tasks; the remote communication path and the local data path share the hardware port on the data loading engine that is connected to the global storage unit. The remote communication path is used to preempt the hardware port according to the preemption time interval configured in the third register; or, to preempt the hardware port according to the preemption time interval specified by the time interval instruction.

16. The first graphics processor according to any one of claims 1 to 7, characterized in that, The data loading engine also includes a local data path, which is used to read and write data in the global storage unit when the multiprocessor performs computing tasks. The remote communication path is used to monitor the local data path and determine the frequency of sending sub-load requests to the global storage unit according to the number of read and write requests on the local data path. The sub-load request is used to read a portion of the target data from the global storage unit.

17. The first graphics processor according to any one of claims 1 to 7, characterized in that, The remote communication path is used to determine the time interval for sending a sub-load request to the global storage unit according to the cross-card bandwidth requirements between the first graphics processor and the second graphics processor. The sub-load request is used to read a portion of the target data from the global storage unit.

18. A remote communication method, characterized in that, The method is applied to a first graphics processor, which includes a multiprocessor, a data loading engine, and a global storage unit. The data loading engine includes a remote communication path. The remote communication channel receives remote communication commands sent by the multiprocessor; In response to the remote communication command, the remote communication channel reads the target data from the global storage unit and sends the target data directly to the second graphics processor without pausing.

19. A chip, characterized in that, The chip includes a first graphics processor as described in any one of claims 1 to 17.

20. A server, characterized in that, The server includes a first graphics processor as described in any one of claims 1 to 17.

21. An electronic device, characterized in that, The electronic device includes a first graphics processor as described in any one of claims 1 to 17.

Images