Data transmission method and device, related equipment, storage medium and computer program product

Abstract

The invention discloses a data transmission method and device, a first graphics processing unit (GPU), a second GPU, a storage medium and a computer program product. The method comprises the steps that a first GPU obtains multiple pieces of first information, the first information comprises data and corresponding address information, the multiple pieces of first information correspond to a second GPU, the second GPU is used for writing the data into a storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU; and generating a first message by using the plurality of pieces of first information, and sending the first message to the second GPU, the first message comprising the plurality of pieces of first information.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data transmission method, apparatus, related equipment, storage medium and computer program product. Background Art

[0002] In the related art, by interconnecting multiple graphics processing units (GPUs), efficient parallel computing can be achieved when processing artificial intelligence (AI) tasks.

[0003] However, in a scenario where multiple GPUs are interconnected, the data transmission efficiency between GPUs is low. Summary of the invention

[0004] To solve the related technical problems, the embodiments of the present application provide a data transmission method, apparatus, related equipment, storage medium and computer program product.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present application embodiment provides a data transmission method, which is applied to a first GPU, including:

[0007] Acquire a plurality of first information, wherein the first information includes data and corresponding address information, the plurality of first information corresponds to a second GPU, the second GPU is used to write the data into a storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU;

[0008] A first message is generated by using the multiple first information, and the first message is sent to the second GPU, where the first message includes the multiple first information.

[0009] In the above solution, the first message further includes second information and third information, the second information is used to represent that the first message is associated with a plurality of first information, and the second information is used to identify the second GPU.

[0010] In the above solution, the header of the first message includes the second information and the third information, and the header of the first message further includes fourth information, and the fourth information includes address information corresponding to the multiple first information.

[0011] In the above solution, the step of generating a first message by using the plurality of first information includes:

[0012] When the sum of the lengths of the data in the acquired multiple first information reaches a preset threshold, the first message is generated using the multiple first information.

[0013] In the above solution, the obtaining of multiple first information includes:

[0014] Acquire a plurality of fifth information, wherein the fifth information includes data and corresponding address information, and the plurality of fifth information corresponds to the second GPU;

[0015] Based on a preset strategy, determining a transmission order of the plurality of fifth information;

[0016] The plurality of first information are selected from the plurality of fifth information according to the transmission order.

[0017] In the above solution, the obtaining of multiple first information includes:

[0018] Acquire a plurality of sixth information, the sixth information including data and corresponding address information, the plurality of sixth information corresponding to a plurality of GPUs, the plurality of GPUs including the second GPU;

[0019] A plurality of sixth information corresponding to the second GPU among the plurality of sixth information is used as the plurality of first information.

[0020] In the above solution, the multiple GPUs include one or more third GPUs except the second GPU, and the method further includes:

[0021] For each third GPU in the one or more third GPUs, multiple sixth information corresponding to the third GPU is determined from the multiple sixth information, and a third message is generated using the determined multiple sixth information, and the third message is reported to the third GPU, wherein the third message includes the determined multiple sixth information.

[0022] The embodiment of the present application also provides a data transmission method, which is applied to a second GPU, comprising:

[0023] receiving a first message sent by a first GPU, where the first message is generated based on a plurality of first information, where the first information includes data and corresponding address information, and the first GPU is interconnected with the second GPU;

[0024] Parsing the first message to obtain multiple first information;

[0025] For each first information of the plurality of first information, data is written into a storage space indicated by the corresponding address information.

[0026] In the above solution, the first message further includes second information and third information, the second information represents that the first message is associated with a plurality of first information, and the second information is used to identify the second GPU.

[0027] In the above solution, the header of the first message includes the second information and the third information, and the header of the first message further includes fourth information, and the fourth information includes address information corresponding to the multiple first information.

[0028] In the above solution, one or more messages are received within a preset time period, and the one or more messages include the first message. The method further includes:

[0029] Parsing the one or more messages to obtain a plurality of seventh information, wherein the seventh information includes data and corresponding address information, and the plurality of seventh information includes the plurality of first information;

[0030] For each piece of address information, the latest seventh information corresponding to the address information is determined, and the data in the latest seventh information is written into the storage space indicated by the address information.

[0031] The embodiment of the present application further provides a data transmission device, applied to a first GPU, comprising:

[0032] an acquisition unit, configured to acquire a plurality of first information, wherein the first information includes data and corresponding address information, the plurality of first information corresponds to a second GPU, the second GPU is configured to write the data into a storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU;

[0033] A sending unit is used to generate a first message using the multiple first information, and send the first message to the second GPU, wherein the first message includes the multiple first information.

[0034] The embodiment of the present application further provides a data transmission device, applied to a second GPU, comprising:

[0035] a receiving unit, configured to receive a first message sent by a first GPU, wherein the first message is generated based on a plurality of first information, the first information includes data and corresponding address information, and the first GPU is interconnected with the second GPU;

[0036] A parsing unit, configured to parse the first message to obtain a plurality of first information;

[0037] The storage unit is used to write data into the storage space indicated by the corresponding address information for each first information of the plurality of first information.

[0038] The embodiment of the present application further provides a first GPU, comprising: a first processor and a first communication interface; wherein,

[0039] The first processor is used to obtain multiple first information through the first communication interface, the first information includes data and corresponding address information, the multiple first information corresponds to the second GPU, the second GPU is used to write the data into the storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU; and use the multiple first information to generate a first message, and send the first message to the second GPU through the first communication interface, the first message includes the multiple first information.

[0040] The embodiment of the present application further provides a second GPU, including:

[0041] A second communication interface, configured to receive a first message sent by a first GPU, where the first message is generated based on a plurality of first information, where the first information includes data and corresponding address information, and the first GPU is interconnected with the second GPU;

[0042] The second processor is used to parse the first message to obtain multiple first information; and for each first information of the multiple first information, write data into the storage space indicated by the corresponding address information.

[0043] The embodiment of the present application further provides a first GPU, comprising: a first processor and a first memory for storing a computer program that can be run on the processor,

[0044] Wherein, when the first processor is used to run the computer program, it executes the steps of any one of the above-mentioned methods on the first GPU side.

[0045] The embodiment of the present application further provides a second GPU, including: a second processor and a second memory for storing a computer program that can be run on the processor,

[0046] Wherein, when the second processor is used to run the computer program, it executes the steps of any method on the second GPU side.

[0047] An embodiment of the present application also provides a storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the steps of any of the above-mentioned methods on the first GPU side, or implements the steps of any of the above-mentioned methods on the second GPU side.

[0048] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods on the first GPU side, or implements the steps of any of the above-mentioned methods on the second GPU side.

[0049] The data transmission method, apparatus, related equipment, storage medium and computer program product provided by the embodiment of the present application, the first GPU obtains multiple first information, the first information includes data and corresponding address information, the multiple first information corresponds to the second GPU, the first GPU and the second GPU are interconnected; the first GPU uses the multiple first information to generate a first message, and sends the first message to the second GPU, the first message includes the multiple first information; the second GPU receives the first message sent by the first GPU; the second GPU parses the first message to obtain multiple first information; the second GPU writes data to the storage space indicated by the corresponding address information for each first information of the multiple first information. The scheme provided by the embodiment of the present application, when the first GPU transmits data to the second GPU, the message sent includes multiple first information (which can also be understood as information corresponding to an access storage operation), so that, compared with the data transmission method in which each message only includes one first information, the proportion of the data part in each message (that is, the proportion of the effective load) is larger, which can improve the data transmission efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a flow chart of the first data transmission method according to an embodiment of the present application;

[0051] Figure 2 This is a flow chart of a second data transmission method according to an embodiment of the present application;

[0052] Figure 3 This is a flow chart of a third data transmission method according to an embodiment of the present application;

[0053] Figure 4 This is a schematic diagram of a data access system for interconnecting multiple GPU cards as an example of the application of this application;

[0054] Figure 5 A flowchart of a GPU remote writing method for application example of this application;

[0055] Figure 6 This is a schematic diagram of the structure of a data transmission device according to an embodiment of the present application;

[0056] Figure 7 This is a schematic diagram of another data transmission device structure according to an embodiment of the present application;

[0057] Figure 8 This is a schematic diagram of the structure of the first GPU in an embodiment of the present application;

[0058] Fig. 9 This is a schematic diagram of the structure of the second GPU in the embodiment of the present application;

[0059] Fig.10This is a schematic diagram of the structure of the data transmission system according to an embodiment of the present application. DETAILED DESCRIPTION

[0060] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0061] In an intelligent computing center (also referred to as an intelligent computing center, or an AI data center), efficient parallel computing can be achieved by establishing interconnections between multiple GPUs (also understood as multi-GPU interconnection, or multi-GPU data interconnection).

[0062] In the related art, data transmission between multiple GPUs mainly relies on the peer-to-peer (P2P) access to storage space (hereinafter referred to as memory access) mechanism (also understood as active peer-to-peer memory access mechanism). In the P2P memory access mechanism, multiple GPUs can directly exchange data, and the data does not need to be processed by the central processing unit (CPU) during transmission, thereby significantly improving the efficiency of data transmission. At the same time, when data is transmitted between GPUs based on the P2P memory access mechanism, the GPU can perform cross-GPU data read and write operations more efficiently. Among them, memory access operations can be performed between GPUs through memory access instructions such as load instructions, store instructions, and atomic instructions.

[0063] For example, suppose that the intelligent computing center contains multiple interconnected GPUs, and one of the multiple GPUs (also understood as a local GPU, hereinafter referred to as a sending GPU) needs to send data to one or more other GPUs (also understood as at least one) (also understood as a remote GPU, hereinafter referred to as a receiving GPU) (i.e., data needs to be written to the video memory of the receiving GPU). At this time, the sending GPU can generate request information based on the P2P memory access mechanism, so that the sending GPU generates a load instruction when it recognizes the request information, and then loads the data to be sent from the video memory of the sending GPU (also understood as local video memory) into the cache for calculation and processing according to the load instruction, and sends the calculated and processed data to the receiving GPU through a communication link based on the P2P memory access mechanism (also understood as a P2P communication link); the sending GPU can also send a store instruction to the receiving GPU to remotely instruct the receiving GPU to write (also understood as store) the data corresponding to the store instruction into the video memory of the receiving GPU, thereby realizing remote data update.

[0064] At the same time, in addition to load and store instructions, in the scenario of multi-GPU interconnection, multiple GPUs can also operate on data through atomic instructions to ensure the atomicity of the operation and the consistency of memory, and avoid memory data errors that may be caused when multiple GPUs (also understood as multiple GPU threads) operate on the same data at the same time. At this time, each GPU maintains its own internal cache consistency.

[0065] From the above description, it can be seen that when multiple GPUs perform data transmission between GPUs based on the P2P memory access mechanism, there is no need for the CPU to intervene in the data transmission process, which can reduce the delay of data transmission and allow computing tasks to be executed in parallel with data transmission.

[0066] However, even if an application designed based on a P2P memory access mechanism (which can also be understood as a data memory access application, or a data memory access program) enables data transmission (which can also be understood as communication between GPUs) and data processing (which can also be understood as calculation) to be performed simultaneously, the system equipped with the data memory access application (which can also be understood as a data memory access system) will still perform poorly due to low data transmission efficiency.

[0067] In the scenario of multiple GPUs interconnected, the reasons for the low efficiency of data transmission between GPUs are mainly the following two aspects:

[0068] First, the communication bandwidth between GPUs is significantly lower than the connection bandwidth between the cache and video memory in the GPU. Therefore, even if complex optimization is performed on programs that apply data access technology between multiple GPUs (also understood as multi-GPU programs), it is usually difficult to achieve the performance of programs that only involve single GPU data access (also understood as single GPU programs). In other words, the bottleneck effect of the communication bandwidth between GPUs limits the performance expansion of systems that apply data access technology between multiple GPUs (also understood as multi-GPU systems, or multi-GPU data access systems, on which the above multi-GPU programs can run).

[0069] Second, the communication bandwidth between GPUs is not fully utilized. Specifically, when accessing data within a single GPU, a single GPU can use the network on chip (NoC) capability to merge multiple access requests to adjacent video memory addresses (also known as access merging), thereby reducing the number of accesses. However, since hardware consistency is usually not supported between multiple GPUs, in the scenario where one GPU (i.e., the sending GPU) among multiple GPUs accesses data in other GPUs (i.e., the receiving GPU) multiple times, the receiving GPU will not merge multiple access requests. At this time, the sending GPU needs to send a message (also known as sending a data packet, specifically including the data to be accessed and the address information of the data), for each access request (such as a write operation), that is, each message can only correspond to one access request, which will result in multiple GPUs accessing data between GPUs based on the P2P memory access mechanism (i.e., using load instructions, store instructions, and atomic instructions for data access). The granularity of the message transmitted between GPUs each time is very small, that is, the amount of data carried by each message is very small, and may even be less than a cache line. At this time, each message contains only a small effective load (i.e., the amount of data, such as 32 bytes), but needs to contain a large protocol overhead (i.e., the transmission resources occupied by the protocol, such as frame-level and link-level protocol overhead) and address information (such as the address information of the receiving GPU, which may occupy 48-64 bits of space), which will lead to low data transmission efficiency (which can also be understood as interconnection efficiency, or bus (which can also be understood as the transmission path resources between GPUs) resource utilization); at the same time, the amount of data carried by each message may be different (which can also be understood as irregularity in data access), which will cause the receiving GPU to determine the corresponding data amount for each message, increasing the complexity of data transmission and low data transmission efficiency.

[0070] From the above description, it can be seen that in the scenario of multi-GPU interconnection, how to improve the efficiency of data transmission is a problem that needs to be solved urgently.

[0071] Based on this, in various embodiments of the present application, when the first GPU transmits data to the second GPU, the sent message contains multiple first information (which can also be understood as information corresponding to an access storage operation). In this way, compared with the data transmission method in which each message only contains one first information, the proportion of the data part in each message (that is, the proportion of the effective load) is larger, thereby improving the data transmission efficiency.

[0072] The present application embodiment provides a data transmission method, which is applied to a first GPU, such as Figure 1 As shown, the method includes:

[0073] Step 101: Acquire a plurality of first information, wherein the first information includes data and corresponding address information, the plurality of first information corresponds to a second GPU, the second GPU is used to write data into a storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU;

[0074] Step 102: Generate a first message using the multiple first information, and send the first message to the second GPU, where the first message includes the multiple first information.

[0075] Here, in actual application, the method can be applied to an intelligent computing center, wherein the intelligent computing center includes multiple GPUs, wherein the multiple GPUs include at least the first GPU and the second GPU, and each GPU in the multiple GPUs is interconnected with other GPUs, that is, each GPU among the multiple GPUs can directly transmit data with other GPUs. In other words, the first GPU can directly transmit data (such as sending messages) with the second GPU.

[0076] When the first GPU sends a message to the second GPU, the first GPU may also be called a sending-end GPU, and the second GPU may also be called a receiving-end GPU. In this embodiment of the present application, the names of the first GPU and the second GPU are not limited as long as their functions are realized.

[0077] In actual application, the first GPU and the second GPU can be used to jointly execute an application (such as a program for executing AI tasks deployed in an intelligent computing center, which can be a large model application). In this case, the compiler of the first GPU can compile the application to obtain multiple operation instructions, and execute the operation instructions through the thread on the first GPU to obtain the data and address information corresponding to the operation instructions. Among them, the operation instructions may include operation instructions for the storage space of the first GPU, and may also include operation instructions for the storage space of the second GPU.

[0078] In the case where the operation instruction includes an operation instruction for the storage space of the second GPU, the first GPU needs to perform an operation (such as a memory access operation) on the storage space of the second GPU (such as the video memory of the second GPU). At this time, the first GPU needs to send the data corresponding to the operation instruction (such as the memory access operation instruction) (which can also be understood as the data corresponding to the operation) and the address information corresponding to the data to the second GPU, so that the second GPU can use the received data and address information to perform corresponding operations, such as storing the data in the storage space indicated by the address information.

[0079] In actual application, after the first GPU obtains the operation instruction for the storage space of the second GPU, it can specifically perform the following operations: the first GPU loads the data corresponding to the operation instruction from the video memory of the first GPU into the cache of the first GPU, and then uses the computing unit of the first GPU to perform calculations on the data loaded into the cache to obtain the data corresponding to the operation instruction that needs to be transmitted to the second GPU (which can also be understood as data to be transmitted, hereinafter referred to as data), and sends the data and address information corresponding to the operation instruction to the second GPU, so that the GPU can use the data and address information to perform the corresponding operation. Here, the operation instruction corresponds to the data one by one.

[0080] Specifically, the first GPU may send the data and address information to the second GPU by sending a message, that is, the first GPU sends a message to the second GPU, and the message includes the data and address information.

[0081] In actual application, when the first GPU needs to perform multiple operations on the storage space of the second GPU, the first GPU needs to send data corresponding to multiple operation instructions and address information corresponding to each data to the second GPU. At this time, each message contains not only data, but also address information corresponding to the data, relevant information of the second GPU and other information, wherein the other information may specifically include protocol information, such as protocol information used to enable the second GPU to successfully receive and process the message. Therefore, if each message sent by the first GPU to the second GPU contains only one data (i.e., only corresponds to one operation instruction), the proportion of the data part in the message (i.e., the proportion of the effective load) is low, that is, when the first GPU transmits data to the second GPU through such a message, the address information and protocol information occupy a large amount of transmission resources (which can also be understood as generating a large amount of overhead), which will result in low efficiency of data transmission.

[0082] Based on this, in the embodiment of the present application, when the first GPU transmits data to the second GPU via a message, each message (such as the first message) contains multiple data. Here, a message containing only one data can be called a small message, and a message containing multiple data can be called a large message. It can be seen that compared with a small message, a large message contains more data and has a higher data transmission efficiency.

[0083] In actual application, before generating a message to be sent to the second GPU, the first GPU needs to first determine which data can be used together to generate a message, that is, which data can be transmitted by the same message.

[0084] Specifically, the data acquired by the first GPU may correspond to multiple GPUs, that is, the first GPU may contain operation instructions corresponding to multiple other GPUs. At this time, after the first GPU acquires the data, it is necessary to classify the data according to the GPU to which the data corresponds (that is, divide the data according to the GPU that receives the data), among which only the data corresponding to the second GPU can be used to generate a message sent to the second GPU.

[0085] Based on this, in one embodiment, the obtaining of multiple first information includes:

[0086] Acquire a plurality of sixth information, the sixth information including data and corresponding address information, the plurality of sixth information corresponding to a plurality of GPUs, the plurality of GPUs including the second GPU;

[0087] A plurality of sixth information corresponding to the second GPU among the plurality of sixth information is used as the plurality of first information.

[0088] Here, the sixth information can also be understood as all data acquired by the first GPU and the address information corresponding to the data. These sixth information may correspond to multiple GPUs, and the embodiment of the present application does not limit the name of the sixth information. The multiple first information refers to the multiple data and corresponding address information contained in the message sent by the first GPU to the second GPU, and the embodiment of the present application does not limit the name of the first information.

[0089] After classifying the acquired sixth information according to the corresponding GPU, the first GPU can use the multiple data and corresponding address information corresponding to the second GPU to generate a message for transmitting data to the second GPU. However, in actual application, not every data corresponding to the second GPU can be directly transmitted to the second GPU through a large message. For example, when there is a logical relationship between certain data and other data, when the first GPU transmits these data, the transmission order of the data needs to match the logical relationship. That is to say, even if the first GPU obtains certain data first, it may need to transmit other data before it can transmit the data. The first GPU cannot generate a message using the data, which will result in a decrease in the efficiency of generating messages and the efficiency of data transmission. Among them, the data with a logical relationship refers to the multiple data obtained by the GPU according to multiple operation instructions with a logical relationship; if there is a certain operation instruction that needs to wait for another operation instruction to be executed before it can be executed in a logical relationship, etc., these operation instructions are operation instructions with a logical relationship.

[0090] For example, assume that each message sent by the first GPU to the second GPU needs to contain 5 data, the first GPU has already obtained 4 data, and there is a logical relationship between the 5th data obtained by the first GPU and another data, the logical relationship indicates that the execution order of the operation instruction corresponding to the 5th data is after the execution order of the operation instruction corresponding to the other data, and at the same time, the first GPU has not yet obtained the other data (the specific reason may include the delay difference between threads, etc.). In this case, when the first GPU transmits data to the second GPU, if the message is directly generated using the 5th data and the 4 data that have been obtained and the message is sent to the second GPU, the logical relationship between the data will be destroyed, which may cause the second GPU to execute the operation instruction corresponding to the 5th data first, and then execute the operation instruction corresponding to another data that has a logical relationship with the 5th data, resulting in the destruction of data dependency. It can be seen that in order for the second GPU to accurately determine the execution order of the operation instruction, the first GPU needs to wait until the next data is obtained before it can generate a message using the next data obtained and the 4 data that have been obtained. It can be seen that the logical relationship between the data may affect the rate at which the first GPU generates messages and reduce the transmission efficiency.

[0091] Based on this, in an embodiment of the present application, the first GPU can adjust the transmission order of the acquired multiple data under the premise of considering data dependency, such as adjusting some data acquired earlier and logically required to be transmitted later to be transmitted earlier, so as to avoid the earlier acquired data not being able to be sent in time, reduce the waiting time for generating messages, and thus improve the transmission efficiency. Here, adjusting the transmission order of multiple data can also be understood as reordering.

[0092] Specifically, the first GPU may adjust the transmission order of the plurality of data according to the characteristics of the GPU. The characteristics of the GPU may specifically include the following two items:

[0093] 1) Weak memory model (also known as weak memory order model, or weak memory model) characteristics: GPUs usually adopt a weak memory model, which means that the order between multiple memory operations (also known as memory access operations, such as read operations, write operations, etc.) executed on the GPU is not always strict. Therefore, the GPU compiler and hardware can reorder the instructions corresponding to the memory operations based on the characteristics of the weak memory model, and the corresponding threads will execute the reordered instructions. Here, reordering instructions means changing the execution order of instructions (such as advancing some logically later instructions) while ensuring data dependencies. This change will not affect the execution results within a single thread, and can reduce the idle and waiting time of GPU resources and improve execution efficiency.

[0094] For example, assume that the GPU includes two threads, namely thread 1 and thread 2, and thread 1 and thread 2 share variables A and variable B (that is, thread 1 and thread 2 can both operate on variables A and variable B), and thread 1 needs to execute a write operation instruction (hereinafter referred to as instruction 1) on variable A before thread B executes a read operation instruction (hereinafter referred to as instruction 2) on variable B. However, since the order between instruction 1 and instruction 2 does not destroy the data dependency, the GPU compiler can reorder the two instructions, that is, execute instruction 2 first and then execute instruction 1. In this way, when the compiler compiles instruction 2 first and then obtains instruction 1, thread 2 can execute instruction 2 first without waiting for thread 1 to complete the execution of instruction 1 obtained later, which can significantly improve the overall execution efficiency of the GPU.

[0095] 2) Spatiotemporal locality (also understood as time locality and spatial locality): Spatiotemporal locality refers to the locality of the data access pattern adopted by the application when accessing the GPU data. Specifically, spatial locality refers to the tendency of the application to access adjacent memory addresses, that is, if a memory address is accessed by the application, other memory addresses near the memory address are likely to be accessed by the application in the near future; temporal locality refers to the tendency of the application to repeatedly access the same memory address, that is, if a memory address is accessed by the application. The memory address is likely to be accessed by the application again in the near future. In actual applications, large model applications usually operate within a larger GPU memory range, that is, the GPU usually has good spatiotemporal locality.

[0096] It can be seen from the above description that when the first GPU needs to perform multiple operations on the storage space of the second GPU, the first GPU tends to continuously access the same video memory address in the second GPU, and / or access adjacent video memory addresses in the second GPU. At this time, the compiler and hardware of the first GPU can adjust the execution order of multiple operation instructions corresponding to the multiple operations (i.e., reorder them) based on the weak memory model characteristics of the GPU without affecting the execution result, and determine the transmission order of the data corresponding to the operation instructions according to the adjusted execution order of the operation instructions. In this way, when the first GPU generates a message, it does not need to strictly abide by the logical order between the data, and can generate the message more flexibly. At the same time, the order between the data will not affect the execution result on the second GPU side.

[0097] Based on this, in one embodiment, the obtaining of multiple first information includes:

[0098] Acquire a plurality of fifth information, wherein the fifth information includes data and corresponding address information, and the plurality of fifth information corresponds to the second GPU;

[0099] Based on a preset strategy, determining a transmission order of the plurality of fifth information;

[0100] The plurality of first information are selected from the plurality of fifth information according to the transmission order.

[0101] Here, the fifth information can also be understood as the data associated with the second GPU and the corresponding address information obtained by the first GPU. The embodiment of the present application does not limit the name of the fifth information; the process of selecting multiple first information from the multiple fifth information can also be understood as selecting data with a first transmission order (i.e., data that can be transmitted through the same large message) according to the reordering result; the preset strategy can also be called a reordering strategy, and the preset strategy can be specifically related to the weak memory model and spatiotemporal local characteristics of the GPU. The embodiment of the present application does not limit the specific name of the preset strategy.

[0102] After determining the plurality of first information, the first GPU may generate a first message using the acquired first information. The first GPU may generate the first message when the number of the acquired first information reaches a preset threshold. That is, after acquiring a preset number of first information, the first GPU generates a first message using the acquired first information and sends the first message to the second GPU.

[0103] Of course, the first GPU can also generate the first message using the acquired first information after the sum of the lengths of the data in the acquired first information reaches a preset threshold, and send the first message to the second GPU. In this way, the data length corresponding to the first message is fixed, and after the second GPU receives the first message, it does not need to confirm the data length of the message, which can simplify the processing flow and improve the efficiency of data transmission.

[0104] Based on this, in one embodiment, the step of generating a first message by using the plurality of first information includes:

[0105] When the sum of the lengths of the data in the acquired multiple first information reaches a preset threshold, the first message is generated using the multiple first information.

[0106] In actual application, the first message generated by the first GPU may include other information in addition to the multiple first information, so that the second GPU can successfully receive the first message and correctly process the first message.

[0107] Specifically, in one embodiment, the first message may further include second information and third information, the second information is used to represent that the first message is associated with a plurality of first information, and the second information is used to identify the second GPU.

[0108] Here, the second information may also be referred to as indication information, and the second information is used to inform the second GPU that the first message contains multiple first information, so that the second GPU can correctly parse the first message. The embodiment of the present application does not limit the name of the second information; the third information may also be referred to as identification information of the second GPU. When the first GPU sends the first message, it can accurately determine the port to which the first GPU and the second GPU are connected according to the third information, and transmit the first message through the determined port. The embodiment of the present application does not limit the name of the third information.

[0109] In actual application, when the first GPU generates the first message, the data of the multiple first information may be included in the data part of the first message, wherein the positional relationship of the data of each first information in the data part is related to the transmission order of the first information; at the same time, the address information, second information and third information corresponding to the data in the first information may be included in the header of the first message. That is to say, in one embodiment, the header of the first message includes the second information and the third information, and the header of the first message also includes fourth information, and the fourth information includes the address information corresponding to the multiple first information. The embodiment of the present application does not limit the name of the fourth information. Of course, the header of the first message may also include the correspondence between the data and the address information, so that the second GPU can determine which address information each data corresponds to based on the correspondence.

[0110] In actual application, when the operation instruction in the first GPU corresponds to multiple GPUs, the first GPU may generate and send corresponding messages to other GPUs in addition to generating and sending the first message to the second GPU.

[0111] Based on this, in one embodiment, the plurality of sixth information acquired by the first GPU corresponds to a plurality of GPUs, and the plurality of GPUs include one or more third GPUs except the second GPU, and the method may further include:

[0112] For each third GPU in the one or more third GPUs, multiple sixth information corresponding to the third GPU is determined from the multiple sixth information, and a third message is generated using the determined multiple sixth information, and the third message is reported to the third GPU, wherein the third message includes the determined multiple sixth information.

[0113] Accordingly, the embodiment of the present application also provides a data transmission method, which is applied to the second GPU, such as Figure 2 As shown, the method includes:

[0114] Step 201: receiving a first message sent by a first GPU, where the first message is generated based on a plurality of first information, where the first information includes data and corresponding address information, and the first GPU is interconnected with the second GPU;

[0115] Step 202: parsing the first message to obtain a plurality of first information;

[0116] Step 203: For each first information of the plurality of first information, write data into a storage space indicated by the corresponding address information.

[0117] Here, in actual application, the first GPU can obtain multiple first information according to the operation instruction, and use the multiple first information to generate and send the first message to the second GPU. The above process has been described in detail above and will not be repeated here.

[0118] After the second GPU receives the first message, in step 202, the second GPU may parse the first message to obtain each first information and determine the relationship between the multiple first information contained in the first message. Specifically, the second GPU may determine the transmission order between the multiple first information according to the location information of the data of the multiple first information contained in the data part of the first message. The specific implementation method of the second GPU parsing the first message can be set according to actual needs, and the embodiment of the present application does not limit this.

[0119] After obtaining multiple first information and the transmission order of the first information, in step 203, the second GPU can execute the operation corresponding to the first information for each first information according to the transmission order, that is, write the data in the first information into the storage space indicated by the corresponding address information.

[0120] Specifically, for each first information, the second GPU uses the address information included in the first information to determine a physical address in the video memory of the second GPU to ensure that the data is written to the correct location of the video memory; after the second GPU applies for and obtains control of the bus (which can be expressed as BUS in English), the data in the first information is transmitted to the corresponding video memory physical address through the bus. Specifically, the video memory may include high bandwidth memory (HBM).

[0121] In actual application, if operations corresponding to multiple first information are associated with the same storage space, the operation corresponding to the first information with a later transmission order among the multiple first information will overwrite the operation corresponding to the first information with an earlier transmission order.

[0122] Exemplarily, it is assumed that the first message includes two first information, namely information A and information B, the transmission order of information B is after information A, and the operation corresponding to information A is to write data M into the storage space corresponding to address O, and the operation corresponding to information B is to write data N into the storage space corresponding to address O. At this time, the second GPU first executes the operation corresponding to information A according to the transmission order, and then executes the operation corresponding to information B. The data stored in the storage space corresponding to address O is data N, that is, the operation corresponding to information A is overwritten.

[0123] Based on this, the second GPU can execute only the latest operation for the same address information within a preset period of time, thereby reducing the occupancy of the bus when writing to the video memory and improving execution efficiency.

[0124] Specifically, in one embodiment, one or more messages are received within a preset time period, and the one or more messages include the first message. The method may further include:

[0125] Parsing the one or more messages to obtain a plurality of seventh information, wherein the seventh information includes data and corresponding address information, and the plurality of seventh information includes the plurality of first information;

[0126] For each piece of address information, the latest seventh information corresponding to the address information is determined, and the data in the latest seventh information is written into the storage space indicated by the address information.

[0127] Accordingly, the embodiment of the present application also provides a data transmission method, such as Figure 3 As shown, the method includes:

[0128] Step 301: a first GPU acquires a plurality of first information, where the first information includes data and corresponding address information, the plurality of first information corresponds to a second GPU, and the first GPU is interconnected with the second GPU;

[0129] Step 302: The first GPU generates a first message using the multiple first information, and sends the first message to the second GPU, where the first message includes the multiple first information;

[0130] Step 303: The second GPU receives the first message sent by the first GPU;

[0131] Step 304: The second GPU parses the first message to obtain a plurality of first information;

[0132] Step 305: The second GPU writes data into the storage space indicated by the corresponding address information for each first information of the plurality of first information.

[0133] Here, it should be noted that the specific processing processes of the first GPU and the second GPU have been described in detail above and will not be repeated here.

[0134] In the data transmission method provided by the embodiment of the present application, the first GPU obtains multiple first information, the first information includes data and corresponding address information, the multiple first information corresponds to the second GPU, and the first GPU is interconnected with the second GPU; the first GPU uses the multiple first information to generate a first message, and sends the first message to the second GPU, the first message includes the multiple first information. The second GPU receives the first message sent by the first GPU, and the second GPU parses the first message to obtain multiple first information; the second GPU writes the data into the storage space indicated by the corresponding address information for each first information of the multiple first information. In the scheme provided by the embodiment of the present application, when the first GPU transmits data to the second GPU, the message sent includes multiple first information (which can also be understood as information corresponding to an access storage operation). In this way, compared with the data transmission method in which each message only includes one first information, the proportion of the data part in each message (that is, the proportion of the effective load) is larger, which can improve the data transmission efficiency.

[0135] The present application is described in further detail below in conjunction with application examples.

[0136] This application example provides a data access system for interconnecting multiple GPU cards, such as Figure 4 As shown, the system includes at least one sending-end GPU (i.e., the first GPU mentioned above) and one or more receiving-end GPUs (i.e., the second GPU mentioned above). The sending-end GPU includes a local post-calculation data reordering module and a scattered message aggregation module; the receiving-end GPU includes a message separation module and a duplicate address checker.

[0137] The Omni-directional Intelligent Sensing Architecture (OISA) technology is introduced into the system to improve the data access efficiency of the interconnection between multiple GPU cards, especially for the transmission of small data messages (i.e., messages with a small data portion, such as messages corresponding to a single operation instruction). Among them, the OISA technology can enhance the interconnection of the GPU architecture, utilize the weak memory model characteristics of the GPU and the spatiotemporal locality of large model applications within a larger memory range, and improve the efficiency of small data transmission.

[0138] Based on the above system, the application example of this application also provides a GPU remote writing method, such as Figure 5 As shown, the following steps are included:

[0139] Step 501: The computing unit of the GPU at the sending end puts the data in the video memory into the cache line through load, store, atomic and other instructions;

[0140] Step 502: the computing unit of the GPU at the sending end computes the data in the cache line (which can also be understood as local computing), obtains the locally computed data (that is, the data included in the sixth information), and sends the locally computed data to the locally computed data reordering module through the bus;

[0141] Step 503: the local calculated data reordering module of the sending end GPU divides each data according to the receiving end GPU corresponding to the data;

[0142] Step 504: the local calculated data reordering module of the sending-end GPU reorders the data corresponding to each receiving-end GPU;

[0143] Here, in actual application, the local post-calculation data reordering module of the sending-end GPU can reorder the video memory access operations of the receiving-end GPU by the threads in the sending-end GPU for each receiving-end GPU without violating data dependencies, and reorder the data corresponding to the receiving-end GPU according to the reordering results of the video memory access operations. This means that even if some write operations should logically occur first, the GPU can reorder them with other operations according to the internal optimization strategy, reduce waiting time, and improve execution efficiency.

[0144] Step 505: the local calculated data reordering module of the GPU at the sending end sends the reordered data to the scattered message aggregation module;

[0145] Step 506: The scattered message aggregation module of the GPU at the transmitting end aggregates the data corresponding to the multiple write operations into one message (ie, the first message mentioned above), and sends the message to the corresponding GPU at the receiving end;

[0146] Here, in actual application, if multiple threads of the sending GPU write to the remote GPU's remote memory address at the same time (i.e., operate on the receiving GPU's memory at the same time), the scattered message aggregation module will merge the data corresponding to these small-grained write operations into a larger unit to be transmitted (i.e., one message) for transmission, instead of sending them separately through multiple messages. This aggregation reduces the number of messages sent to the network, thereby reducing protocol overhead.

[0147] Specifically, the scattered message aggregation module generates a large data message for each receiving end GPU based on the total length of the data, and the large data message includes multiple data corresponding to the receiving end GPU, the receiving end GPU memory address corresponding to each data, the receiving end GPU identifier (i.e. the above-mentioned second information, such as the receiving end GPU ID) and protocol information (i.e. the above-mentioned third information, which can also be understood as the OISA protocol header). Among them, the header of the large data message includes the GPU identifier, the protocol information, the receiving end GPU memory address corresponding to each data, and the data part of the large data message includes multiple data corresponding to the receiving end GPU.

[0148] Step 507: After receiving the aggregated big data message, the receiving end GPU parses it through the message separation module (which can also be understood as unpacking); and sends the parsed message to the duplicate address checker of the receiving end GPU;

[0149] Here, in actual application, the message separation module removes the header information of the big data message and only retains the video memory address and data of the GPU at the receiving end to optimize the transmission process.

[0150] Step 508: The duplicate address checker of the receiving end GPU identifies and eliminates the write operation data of the duplicate video memory address within a preset time range, and writes the eliminated write operation data into the video memory space of the receiving end GPU.

[0151] Specifically, if a remote GPU memory address is written multiple times, OISA will select only the last written data and delete the previous operations except the last one within the timer range, reducing unnecessary data transmission and improving the utilization of network bandwidth.

[0152] The solution provided by the application example of this application optimizes the data transmission process in the multi-GPU system through the collaborative work of the above modules, especially the transmission of small messages, improves the transmission efficiency and bandwidth utilization, and thus solves the performance bottleneck problem existing in the memory access system.

[0153] Specifically, it includes the following aspects:

[0154] The scattered message aggregation module aggregates small-granularity write operations, reducing protocol overhead and improving transmission efficiency in multi-GPU interconnection scenarios;

[0155] Improved the traditional P2P memory access mechanism, and reduced the efficiency problems caused by small-granularity data transmission through the scattered message aggregation module during message transmission;

[0156] The local post-computation data reordering module allows the GPU to reorder the memory access operations of threads without violating data dependencies, thereby making more efficient use of the GPU's computing resources.

[0157] The duplicate address checker identifies and eliminates the write operation data of the duplicate video memory address, thus reducing the bus bandwidth resources wasted due to the repeated transmission of the data for the same video memory address;

[0158] In continuous access mode, the optimized large data message transmission method improves the local GPU cache system's ability to merge multiple thread accesses, reduces the number of bus and video memory accesses, and improves the cache system's merging capabilities;

[0159] For non-contiguous access patterns, data is processed through a local post-calculation data reordering module and a duplicate address checker, which reduces the performance loss caused by irregular access patterns and improves performance in non-contiguous access patterns.

[0160] The improved solution is consistent with the existing GPU programming model, which is easy for developers to accept and integrate without changing existing programming habits;

[0161] By solving the data transmission bottleneck in multi-GPU systems, the scalability of the system is improved, allowing the system to more effectively handle complex computing tasks involving more GPUs;

[0162] By optimizing the data transmission process and protocol, the bandwidth utilization is improved, making the communication between multiple GPUs more efficient;

[0163] The improved scheme can adapt to different data access modes, whether continuous access or non-continuous access, and can ensure high transmission efficiency;

[0164] By reducing protocol overhead and optimizing the data transmission process, the data transmission delay is reduced and the system response speed is improved.

[0165] In order to implement the method provided by the first GPU side of the embodiment of the present application, the embodiment of the present application also provides a data transmission device, which is arranged on the first GPU, such as Figure 6 As shown, the device comprises:

[0166] An acquisition unit 601 is used to acquire a plurality of first information, where the first information includes data and corresponding address information, the plurality of first information corresponds to a second GPU, the second GPU is used to write the data into a storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU;

[0167] The sending unit 602 is configured to generate a first message using the multiple first information, and send the first message to the second GPU, where the first message includes the multiple first information.

[0168] In one embodiment, the sending unit 602 is specifically configured to:

[0169] When the sum of the lengths of the data in the acquired multiple first information reaches a preset threshold, the first message is generated using the multiple first information.

[0170] In one embodiment, the acquisition unit 601 is specifically configured to:

[0171] Acquire a plurality of fifth information, wherein the fifth information includes data and corresponding address information, and the plurality of fifth information corresponds to the second GPU;

[0172] Based on a preset strategy, determining a transmission order of the plurality of fifth information;

[0173] The plurality of first information are selected from the plurality of fifth information according to the transmission order.

[0174] In one embodiment, the acquisition unit 601 is specifically configured to:

[0175] Acquire a plurality of sixth information, the sixth information including data and corresponding address information, the plurality of sixth information corresponding to a plurality of GPUs, the plurality of GPUs including the second GPU;

[0176] A plurality of sixth information corresponding to the second GPU among the plurality of sixth information is used as the plurality of first information.

[0177] In one embodiment, the multiple GPUs include one or more third GPUs except the second GPU, and the sending unit 602 is further configured to:

[0178] For each third GPU in the one or more third GPUs, multiple sixth information corresponding to the third GPU is determined from the multiple sixth information, and a third message is generated using the determined multiple sixth information, and the third message is reported to the third GPU, wherein the third message includes the determined multiple sixth information.

[0179] In actual application, the acquisition unit 601 and the sending unit 602 can be implemented by a processor in a data transmission device in combination with a communication interface.

[0180] In order to implement the method on the second GPU side of the embodiment of the present application, the embodiment of the present application also provides a data transmission device, which is arranged on the second GPU, such as Figure 7 As shown, the device comprises:

[0181] A receiving unit 701 is configured to receive a first message sent by a first GPU, where the first message is generated based on a plurality of first information, where the first information includes data and corresponding address information, and the first GPU is interconnected with the second GPU;

[0182] A parsing unit 702, configured to parse the first message to obtain a plurality of first information;

[0183] The storage unit 703 is used to write data into the storage space indicated by the corresponding address information for each first information of the multiple first information.

[0184] In one embodiment, one or more messages are received within a preset time period, and the one or more messages include the first message. The parsing unit 702 is further configured to:

[0185] Parsing the one or more messages to obtain a plurality of seventh information, wherein the seventh information includes data and corresponding address information, and the plurality of seventh information includes the plurality of first information;

[0186] The storage unit 703 is further used for:

[0187] For each piece of address information, the latest seventh information corresponding to the address information is determined, and the data in the latest seventh information is written into the storage space indicated by the address information.

[0188] In actual application, the parsing unit 702 and the storage unit 703 may be implemented by a processor in a data transmission device, and the receiving unit 701 may be implemented by a communication interface in the data transmission device.

[0189] It should be noted that: the data transmission device provided in the above embodiment only uses the division of the above program units as an example when performing data transmission. In actual applications, the above processing can be assigned to different program units as needed, that is, the internal structure of the device is divided into different program units to complete all or part of the processing described above. In addition, the data transmission device provided in the above embodiment and the data transmission method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0190] Based on the hardware implementation of the above program module, and in order to implement the method on the first GPU side of the embodiment of the present application, the embodiment of the present application also provides a first GPU, such as Figure 8 As shown, the first GPU 800 includes:

[0191] A first communication interface 801, capable of exchanging information with other devices (such as a second GPU);

[0192] A first processor 802 is connected to the first communication interface 801 to implement information interaction with other devices, and is used to execute the method provided by one or more technical solutions of the first GPU side when running a computer program;

[0193] A first memory 803 , in which the computer program is stored.

[0194] Specifically, the first processor 802 is configured to:

[0195] Acquire multiple first information through the first communication interface 801, the first information includes data and corresponding address information, the multiple first information corresponds to the second GPU, the second GPU is used to write data into the storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU; and generate a first message using the multiple first information, and send the first message to the second GPU through the first communication interface 801, the first message includes the multiple first information.

[0196] In one embodiment, the first processor 802 is specifically configured to:

[0197] When the sum of the lengths of the data in the acquired multiple first information reaches a preset threshold, the first message is generated using the multiple first information.

[0198] The first processor 802 is specifically configured to:

[0199] Acquire a plurality of fifth information through the first communication interface 801, where the fifth information includes data and corresponding address information, and the plurality of fifth information corresponds to the second GPU;

[0200] Based on a preset strategy, determining a transmission order of the plurality of fifth information;

[0201] The plurality of first information are selected from the plurality of fifth information according to the transmission order.

[0202] In one embodiment, the first processor 802 is specifically configured to:

[0203] Acquire a plurality of sixth information through the first communication interface 801, the sixth information including data and corresponding address information, the plurality of sixth information corresponding to a plurality of GPUs, the plurality of GPUs including the second GPU;

[0204] A plurality of sixth information corresponding to the second GPU among the plurality of sixth information is used as the plurality of first information.

[0205] In one embodiment, the multiple GPUs include one or more third GPUs in addition to the second GPU, and the first processor 802 is further configured to:

[0206] For each third GPU in the one or more third GPUs, multiple sixth information corresponding to the third GPU is determined from the multiple sixth information, and a third message is generated using the determined multiple sixth information, and the third message is reported to the third GPU, wherein the third message includes the determined multiple sixth information.

[0207] It should be noted that the specific processing process of the first processor 802 and the first communication interface 801 can be understood by referring to the above method.

[0208] Of course, in actual application, the various components in the first GPU 800 are coupled together through the bus system 804. It can be understood that the bus system 804 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 804 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 8 Various buses are labeled as bus system 804 .

[0209] The first memory 803 in the embodiment of the present application is used to store various types of data to support the operation of the first GPU 800. Examples of such data include: any computer program used to operate on the first GPU 800.

[0210] The method disclosed in the above embodiment of the present application can be applied to the first processor 802, or implemented by the first processor 802. The first processor 802 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware or software instructions in the first processor 802. The above-mentioned first processor 802 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The first processor 802 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the first memory 803, and the first processor 802 reads the information in the first memory 803 and completes the steps of the above method in combination with its hardware.

[0211] In an exemplary embodiment, the first GPU 800 may be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to execute the aforementioned method.

[0212] Based on the hardware implementation of the above program module, and in order to implement the method on the second GPU side of the embodiment of the present application, the embodiment of the present application also provides a second GPU, such as Fig. 9 As shown, the second GPU 900 includes:

[0213] The second communication interface 901 is capable of exchanging information with other devices (such as the first GPU);

[0214] A second processor 902 is connected to the second communication interface 901 to implement information interaction with other devices, and is used to execute the method provided by one or more technical solutions on the second GPU side when running a computer program;

[0215] A second memory 903 , in which the computer program is stored.

[0216] Specifically, the second communication interface 901 is used to:

[0217] receiving a first message sent by a first GPU, where the first message is generated based on a plurality of first information, where the first information includes data and corresponding address information, and the first GPU is interconnected with the second GPU;

[0218] The second processor 902 is configured to:

[0219] Parse the first message to obtain a plurality of first information; and write data into a storage space indicated by the corresponding address information for each of the plurality of first information.

[0220] In one embodiment, one or more messages are received within a preset time period, and the one or more messages include the first message. The second processor 902 is further configured to:

[0221] Parsing the one or more messages to obtain a plurality of seventh information, wherein the seventh information includes data and corresponding address information, and the plurality of seventh information includes the plurality of first information;

[0222] For each piece of address information, the latest seventh information corresponding to the address information is determined, and the data in the latest seventh information is written into the storage space indicated by the address information.

[0223] It should be noted that the specific processing process of the second processor 902 and the second communication interface 901 can be understood by referring to the above method.

[0224] Of course, in actual application, the components in the second GPU 900 are coupled together via the bus system 904. It is understood that the bus system 904 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 904 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig. 9 Various buses are labeled as bus system 904.

[0225] The second memory 903 in the embodiment of the present application is used to store various types of data to support the operation of the second GPU 900. Examples of such data include: any computer program used to operate on the second GPU 900.

[0226] The method disclosed in the above embodiment of the present application can be applied to the second processor 902, or implemented by the second processor 902. The second processor 902 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of the hardware in the second processor 902 or an instruction in the form of software. The above-mentioned second processor 902 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The second processor 902 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the second memory 903, and the second processor 902 reads the information in the second memory 903 and completes the steps of the above method in combination with its hardware.

[0227] In an exemplary embodiment, the second GPU 900 may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general purpose processors, controllers, MCUs, Microprocessors, or other electronic components to perform the aforementioned methods.

[0228] It can be understood that the memory (first memory 803, second memory 903) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory, a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0229] In an exemplary embodiment, the embodiment of the present application further provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a first memory 803 storing a computer program, the computer program can be executed by the first processor 802 of the first GPU 800 to complete the steps of the first GPU side method, and for another example, including a second memory 903 storing a computer program, the computer program can be executed by the second processor 902 of the second GPU 900 to complete the steps of the second GPU side method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM.

[0230] In an exemplary embodiment, the embodiment of the present application also provides a computer program product, including a computer program, which can be executed by the first processor 802 of the first GPU 800 to complete the steps of the aforementioned first GPU side method, or the computer program can be executed by the second processor 902 of the second GPU 900 to complete the steps of the aforementioned second GPU side method.

[0231] In order to implement the method provided in the embodiment of the present application, the embodiment of the present application also provides a data transmission system, such as Fig.10 As shown, the system includes: a first GPU 1001 and a second GPU 1002, wherein the second GPU 1002 includes at least one second GPU.

[0232] Here, it should be noted that the specific processing process of the first GPU 1001 and the second GPU 1002 has been described in detail above and will not be repeated here.

[0233] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0234] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0235] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A data transmission method, characterized in that: Applied to a first graphics processing unit GPU, comprising: Acquire a plurality of first information, wherein the first information includes data and corresponding address information, the plurality of first information corresponds to a second GPU, the second GPU is used to write the data into a storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU; A first message is generated by using the multiple first information, and the first message is sent to the second GPU, where the first message includes the multiple first information.

2. The method according to claim 1, characterized in that The first message further includes second information and third information, the second information is used to represent that the first message is associated with a plurality of first information, and the second information is used to identify the second GPU.

3. The method according to claim 2, characterized in that The header of the first message includes the second information and the third information. The header of the first message further includes fourth information. The fourth information includes address information corresponding to the multiple first information.

4. The method according to any one of claims 1 to 3, characterized in that: The step of generating a first message by using the plurality of first information includes: When the sum of the lengths of the data in the acquired multiple first information reaches a preset threshold, the first message is generated using the multiple first information.

5. The method according to any one of claims 1 to 3, characterized in that: The obtaining of a plurality of first information includes: Acquire a plurality of fifth information, wherein the fifth information includes data and corresponding address information, and the plurality of fifth information corresponds to the second GPU; Based on a preset strategy, determining a transmission order of the plurality of fifth information; The plurality of first information are selected from the plurality of fifth information according to the transmission order.

6. The method according to any one of claims 1 to 3, characterized in that: The obtaining of a plurality of first information includes: Acquire a plurality of sixth information, the sixth information including data and corresponding address information, the plurality of sixth information corresponding to a plurality of GPUs, the plurality of GPUs including the second GPU; A plurality of sixth information corresponding to the second GPU among the plurality of sixth information is used as the plurality of first information.

7. The method according to claim 6, characterized in that The plurality of GPUs include one or more third GPUs other than the second GPU, and the method further includes: For each third GPU in the one or more third GPUs, multiple sixth information corresponding to the third GPU is determined from the multiple sixth information, and a third message is generated using the determined multiple sixth information, and the third message is reported to the third GPU, wherein the third message includes the determined multiple sixth information.

8. A data transmission method, characterized in that: Applied to the second GPU, including: receiving a first message sent by a first GPU, where the first message is generated based on a plurality of first information, where the first information includes data and corresponding address information, and the first GPU is interconnected with the second GPU; Parsing the first message to obtain multiple first information; For each first information of the plurality of first information, data is written into a storage space indicated by the corresponding address information.

9. The method according to claim 8, characterized in that The first message further includes second information and third information, the second information represents that the first message is associated with a plurality of first information, and the second information is used to identify the second GPU.

10. The method according to claim 9, characterized in that The header of the first message includes the second information and the third information. The header of the first message further includes fourth information. The fourth information includes address information corresponding to the multiple first information.

11. The method according to any one of claims 8 to 10, characterized in that: One or more messages are received within a preset time period, the one or more messages including the first message, and the method further includes: Parsing the one or more messages to obtain a plurality of seventh information, wherein the seventh information includes data and corresponding address information, and the plurality of seventh information includes the plurality of first information; For each piece of address information, the latest seventh information corresponding to the address information is determined, and the data in the latest seventh information is written into the storage space indicated by the address information.

12. A data transmission device, characterized in that: Applied to the first GPU, including: an acquisition unit, configured to acquire a plurality of first information, wherein the first information includes data and corresponding address information, the plurality of first information corresponds to a second GPU, the second GPU is configured to write the data into a storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU; A sending unit is used to generate a first message using the multiple first information, and send the first message to the second GPU, wherein the first message includes the multiple first information.

13. A data transmission device, characterized in that: Applied to the second GPU, including: a receiving unit, configured to receive a first message sent by a first GPU, wherein the first message is generated based on a plurality of first information, the first information includes data and corresponding address information, and the first GPU is interconnected with the second GPU; A parsing unit, configured to parse the first message to obtain a plurality of first information; The storage unit is used to write data into the storage space indicated by the corresponding address information for each first information of the plurality of first information.

14. A first GPU, characterized in that: include: A first processor and a first communication interface; wherein, The first processor is used to obtain multiple first information through the first communication interface, the first information includes data and corresponding address information, the multiple first information corresponds to the second GPU, the second GPU is used to write the data into the storage space indicated by the corresponding address information, and the first GPU is interconnected with the second GPU; and use the multiple first information to generate a first message, and send the first message to the second GPU through the first communication interface, the first message includes the multiple first information.

15. A second GPU, characterized in that: include: A second communication interface, configured to receive a first message sent by a first GPU, where the first message is generated based on a plurality of first information, where the first information includes data and corresponding address information, and the first GPU is interconnected with the second GPU; The second processor is used to parse the first message to obtain multiple first information; and for each first information of the multiple first information, write data into the storage space indicated by the corresponding address information.

16. A GPU, characterized in that: include: a first processor and a first memory for storing a computer program executable on the processor, Wherein, when the first processor is used to run the computer program, the steps of the method described in any one of claims 1 to 7 are executed.

17. A GPU, characterized in that: include: a second processor and a second memory for storing a computer program executable on the processor, Wherein, when the second processor is used to run the computer program, the steps of the method described in any one of claims 8 to 11 are executed.

18. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7, or implements the steps of the method according to any one of claims 8 to 11.

19. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7, or implements the steps of the method according to any one of claims 8 to 11.

Images