Data processing method and equipment

Abstract

The invention discloses a data processing method and equipment, and the method comprises the steps: a processor determines to transmit first data on a first path, and obtains the transmission performance information of a first data block size of the first data, the first path is from a first GPU to a second GPU, the first GPU is located at a first NUMA node, the second GPU is located at a second NUMA node, and the first data block size of the first data is smaller than the second data block size of the first data; the first path supports at least two paths, and in different paths, the connection modes of the first NUMA node and the second NUMA node are different; the processor determines a scheduling strategy based on the transmission performance information of the size of the first data block, the scheduling strategy being used for representing a transmission mode of transmitting the first data through the at least two paths; and the processor schedules the first data to be transmitted from the first GPU to the second GPU through the at least two paths based on the scheduling strategy. According to the scheme, the throughput capacity of communication between the GPUs is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and relates to but is not limited to data processing methods and devices. Background Art

[0002] The number of parameters in deep learning (DL) models has exploded, and a single GPU can no longer complete the training task. In order to train such models, parallel training distributes computing and communication requirements to multiple graphics processing units (GPUs), and uses communication between GPUs to share computing results or synchronize training status. Therefore, the performance of communication between GPUs becomes critical to training throughput. Summary of the invention

[0003] The present application provides a data processing method and device, and the solution of the present application improves the throughput capacity of communication between GPUs.

[0004] The technical solution of this application is implemented as follows:

[0005] The present application provides a data processing method, which is applied to an electronic device, wherein the electronic device comprises: a processor; the method comprises:

[0006] The processor determines to transmit first data on a first path, and then obtains transmission performance information of a first data block size of the first data, wherein the first path is from a first GPU to a second GPU, the first GPU is located at a first non-uniform memory access (NUMA) node, the second GPU is located at a second NUMA node, the first path supports at least two paths, and in different paths, the first NUMA node and the second NUMA node are connected in different ways;

[0007] The processor determines a scheduling strategy based on the transmission performance information of the first data block size, wherein the scheduling strategy is used to characterize a transmission mode of transmitting the first data through the at least two paths;

[0008] The processor schedules the first data to be transmitted from the first GPU to the second GPU through the at least two paths based on the scheduling policy.

[0009] In some embodiments, determining a scheduling strategy based on the transmission performance information of the first data block size includes:

[0010] Obtaining a configured scheduling policy associated with the first data block size;

[0011] The configured scheduling strategy is adjusted according to the transmission performance information to obtain the scheduling strategy.

[0012] In some embodiments, the transmission performance information includes first load information, and the method further includes:

[0013] Acquire a first number of second load information of a first thread, where the first thread is a thread that transmits the first data, and the second load information is load information of the first thread when transmitting a data block of the first data block size;

[0014] The first load information is determined based on the first number of the second load information.

[0015] In some embodiments, the at least two pathways include at least two of the following pathways:

[0016] A first path, in which the first GPU is connected to the second GPU via a bounce buffer;

[0017] A second path, in which the first GPU is connected to the second GPU via a non-transparent bridge (NTB);

[0018] A third path, in which the first CPU in the first NUMA node is connected to the second CPU in the second NUMA node through a quick path interconnect (QPI).

[0019] In some embodiments, the scheduling strategy includes a data allocation ratio of each of the at least two paths; and based on the scheduling strategy, scheduling the first data to be transmitted from the first GPU to the second GPU through the at least two paths includes:

[0020] Determine at least one target path among the at least two paths according to a data allocation ratio of each path among the at least two paths, and determine an access address corresponding to each target path among the at least one target path according to the data allocation ratio of each target path among the at least one target path and a storage location of the first data in a memory of the first GPU, wherein the target path is a path used for transmission of the first data;

[0021] For each target path in the at least one target path, based on an access address corresponding to the target path, second data is scheduled to be transmitted from the first GPU to the second GPU through the target path, where the second data is data in the first data transmitted using the target path.

[0022] In some embodiments, determining the access address corresponding to each target path in the at least one target path according to the data allocation ratio of each target path in the at least one target path and the storage location of the first data in the memory of the first GPU includes:

[0023] Determining, according to the size of the first data and the data allocation ratio of each target path in the at least one target path, the data size of the second data used for transmission by each target path in the at least one target path in the first data;

[0024] For each target path in the at least one target path, an access address corresponding to the target path is determined according to a data size of second data used to transmit the target path and a first address of the first data, wherein the first address is a starting address of a storage location of the first data in a memory of the first GPU.

[0025] In some embodiments, the electronic device includes the second NUMA node, and scheduling the second data to be transmitted from the first GPU to the second GPU through the target path based on the access address corresponding to the target path includes:

[0026] Reading the second data from the access address corresponding to the target path, and copying the read second data to a shared memory area, wherein the shared memory area is a mapping of a bounce buffer or an NTB in a CPU memory of a second NUMA node;

[0027] Scheduling the second GPU to read the second data from the shared memory area;

[0028] Schedule the second GPU to copy the second data to the memory of the second GPU.

[0029] In some embodiments, scheduling the second GPU to read the second data from the shared memory area includes:

[0030] scheduling the bounce buffer to send a notification to the second GPU when the second data in the first shared memory area is available, wherein the notification is used to instruct the second GPU to read the second data;

[0031] The second GPU is scheduled to read the second data from the shared memory area in response to the notification.

[0032] In some embodiments, the method further comprises:

[0033] The processor initializes the shared memory in the CPU memory before reading the second data from the access address corresponding to the target path, and creates a mapping from the bounce buffer or the NTB to the shared memory area in the shared memory;

[0034] After scheduling the second GPU to copy the second data to the memory of the second GPU, the processor releases the mapping of the bounce buffer or the NTB to the shared memory area.

[0035] In some embodiments, the electronic device includes the first NUMA node and the second NUMA node, the first GPU and the second GPU run the same model, and the method further includes:

[0036] The second GPU processes the first data based on the running model to obtain a first output result;

[0037] The first GPU processes the third data based on the running model to obtain a second output result;

[0038] The processor schedules the first data output result to be transmitted to the first GPU;

[0039] The first GPU obtains fourth data based on the first output result and the second output result, and the fourth data is used to determine the loss or to update the parameters of the model.

[0040] The present application provides an electronic device, the electronic device comprising a first NUMA node, a second NUMA node, and a processor; wherein the processor is located in the first NUMA node or the second NUMA node, or the processor is independent of the first NUMA node and the second NUMA node; the first NUMA node comprises a first GPU, the second NUMA node comprises a second GPU, a first path from the first GPU to the second GPU supports at least two paths, and in different paths, the first NUMA node and the second NUMA node are connected in different ways;

[0041] The processor is configured to determine that first data is transmitted on a first path, and then obtain transmission performance information of a first data block size of the first data;

[0042] The processor is further configured to determine a scheduling strategy based on the transmission performance information of the first data block size, wherein the scheduling strategy is used to characterize a transmission mode of transmitting the first data through the at least two paths;

[0043] The processor is further configured to schedule, based on the scheduling policy, the transmission of the first data from the first GPU to the second GPU through the at least two paths.

[0044] In some embodiments, the processor is further configured to:

[0045] Obtaining a configured scheduling policy associated with the first data block size;

[0046] The configured scheduling strategy is adjusted according to the transmission performance information to obtain the scheduling strategy.

[0047] In some embodiments, the processor is further configured to:

[0048] Acquire a first number of second load information of a first thread, where the first thread is a thread that transmits the first data, and the second load information is load information of the first thread when transmitting a data block of the first data block size;

[0049] Based on the first number of the second load information, it is determined that the transmission performance information includes first load information.

[0050] In some embodiments, the at least two pathways include at least two of the following pathways:

[0051] a first path, in which the first GPU is connected to the second GPU via a bounce buffer;

[0052] a second path, in which the first GPU is connected to the second GPU via a non-transparent bridge NTB;

[0053] A third path, in which the first CPU in the first NUMA node is connected to the second CPU in the second NUMA node through a quick path interconnect (QPI).

[0054] In some embodiments, the processor is further configured to:

[0055] Determine at least one target path among the at least two paths according to the data allocation ratio of each path among the at least two paths included in the scheduling strategy, and determine an access address corresponding to each target path among the at least one target path according to the data allocation ratio of each target path among the at least one target path and a storage location of the first data in a memory of the first GPU, wherein the target path is a path used for transmission of the first data;

[0056] For each target path in the at least one target path, based on an access address corresponding to the target path, second data is scheduled to be transmitted from the first GPU to the second GPU through the target path, where the second data is data in the first data transmitted using the target path.

[0057] In some embodiments, the processor is further configured to:

[0058] Determining, according to the size of the first data and the data allocation ratio of each target path in the at least one target path, the data size of the second data used for transmission by each target path in the at least one target path in the first data;

[0059] For each target path in the at least one target path, an access address corresponding to the target path is determined according to a data size of second data used to transmit the target path and a first address of the first data, wherein the first address is a starting address of a storage location of the first data in a memory of the first GPU.

[0060] In some embodiments, the processor is further configured to:

[0061] Reading the second data from the access address corresponding to the target path, and copying the read second data to a shared memory area, wherein the shared memory area is a mapping of a bounce buffer or an NTB in a CPU memory of a second NUMA node;

[0062] Scheduling the second GPU to read the second data from the shared memory area;

[0063] Schedule the second GPU to copy the second data to the memory of the second GPU.

[0064] In some embodiments, the processor is further configured to:

[0065] scheduling the bounce buffer to send a notification to the second GPU when the second data in the first shared memory area is available, wherein the notification is used to instruct the second GPU to read the second data;

[0066] The second GPU is scheduled to read the second data from the shared memory area in response to the notification.

[0067] In some embodiments, the processor is further configured to:

[0068] Before reading the second data from the access address corresponding to the target path, initialize the shared memory in the CPU memory and create a mapping from the bounce buffer or the NTB to the shared memory area in the shared memory;

[0069] After scheduling the second GPU to copy the second data to the memory of the second GPU, releasing the mapping of the bounce buffer or the NTB to the shared memory area.

[0070] In some embodiments, the first GPU and the second GPU run the same model;

[0071] The second GPU is used to process the first data based on the running model to obtain a first output result;

[0072] The first GPU is used to process the third data based on the running model to obtain a second output result;

[0073] The processor is configured to schedule the first data output result to be transmitted to the first GPU;

[0074] The first GPU is used to obtain fourth data based on the first output result and the second output result, and the fourth data is used to determine the loss or to update the parameters of the model.

[0075] It can be seen that the first path between GPUs supports multiple channels, and when there is first data to be transmitted on the path, transmission performance information is determined based on the data block size of the first data, and a scheduling strategy for transmitting the first data on multiple channels supported by the first path is determined based on the transmission performance information, and the first data is scheduled to be transmitted from one GPU to another GPU through the multiple channels supported by the first path based on the scheduling strategy; here, the scheduling strategy for transmitting the first data on multiple channels is determined based on the transmission performance information of the current data block size of the first data, and the transmission mode on multiple channels can be effectively controlled according to the transmission performance of the current data block size, and the transmission of the first data can be reasonably scheduled, the transmission capacity of various channels is balanced, and the overall throughput capacity of the first path is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 An optional structural diagram of an electronic device provided in an embodiment of the present application;

[0077] Figure 2 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0078] Figure 3 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0079] FIG. 4A to FIG. 4C Each of them is an optional schematic diagram of a path supported by the first path provided in an embodiment of the present application;

[0080] Figure 5 An optional structural diagram of an electronic device provided in an embodiment of the present application;

[0081] Figure 6 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0082] Figure 7 An optional flowchart of a data processing method provided in an embodiment of the present application;

[0083] Figure 8 An optional schematic diagram of a path provided in an embodiment of the present application;

[0084] Fig. 9 An optional schematic diagram of a path provided in an embodiment of the present application;

[0085] Fig.10 An optional schematic diagram of a virtual path provided in an embodiment of the present application;

[0086] Fig.11 An optional schematic diagram of a connection topology provided in an embodiment of the present application;

[0087] Fig.12 An optional schematic diagram of a communication channel provided in an embodiment of the present application;

[0088] Fig.13 A schematic diagram of an optional flow chart of data transmission provided in an embodiment of the present application;

[0089] Fig.14 A schematic diagram of an optional flow chart of data transmission provided in an embodiment of the present application;

[0090] Fig.15 An optional schematic diagram of updating the scheduling strategy provided in an embodiment of the present application;

[0091] Fig.16 An optional schematic diagram of a model training process provided in an embodiment of the present application;

[0092] Figure 17A-17B Each of them is a schematic diagram of an optional structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0093] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the specific technical solution of the application will be further described in detail below in conjunction with the drawings in the embodiments of the present application. The following embodiments are used to illustrate the present application, but are not used to limit the scope of the present application.

[0094] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0095] In the following description, the terms "first\second\third" are used only as examples to distinguish different objects, and do not represent a specific order for the objects, nor do they have a limitation on the order of precedence. It is understandable that "first\second\third" can be interchanged with a specific order or order of precedence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0096] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0097] The embodiment of the present application can provide a data processing method. In practical applications, the data processing method can be implemented by an antenna control device, and each functional entity in the antenna control device can be collaboratively implemented by hardware resources of the terminal device, such as computing resources such as a processor, and communication resources (such as for supporting various communication methods such as optical cables and cellular).

[0098] In a first aspect, an embodiment of the present application provides a data processing method. Figure 1 The method is applied to an antenna control device; wherein the antenna control device can be deployed in a terminal device.

[0099] Figure 1 A schematic diagram of an optional electronic device is shown. Figure 1 As shown, the electronic device 100 includes a processor 101 .

[0100] The processor 101 is configured to determine that first data is transmitted on a first path, and then obtain transmission performance information of a first data block size of the first data, wherein the first path is from a first GPU to a second GPU, the first GPU is located at a first non-uniform memory access NUMA node, the second GPU is located at a second NUMA node, the first path supports at least two paths, and in different paths, the first NUMA node and the second NUMA node are connected in different ways;

[0101] The processor 101 is configured to determine a scheduling strategy based on the transmission performance information of the first data block size, where the scheduling strategy is used to characterize a transmission mode of transmitting the first data through the at least two paths;

[0102] The processor 101 is configured to schedule, based on the scheduling policy, the transmission of the first data from the first GPU to the second GPU through the at least two paths.

[0103] In a second aspect, the present application provides a data processing method, which can be applied Figure 1 The electronic equipment shown, reference Figure 2 The method may include Figure 2 S201 to S203 shown.

[0104] S201. Determine to transmit first data on a first path, then obtain transmission performance information of a first data block size of the first data, wherein the first path is from a first GPU to a second GPU, the first GPU is located at a first non-uniform memory access NUMA node, the second GPU is located at a second NUMA node, the first path supports at least two paths, and in different paths, the first NUMA node and the second NUMA node are connected in different ways.

[0105] In an embodiment of the present application, a NUMA node includes a CPU and at least one GPU, wherein the first NUMA node and the second NUMA node are different NUMA nodes. At least one GPU is connected to the CPU via a root complex (RC), thereby realizing the connection between at least one GPU and the CPU.

[0106] The first GPU is one of the multiple GPUs included in the first NUMA node. The second GPU is one of the multiple GPUs included in the second NUMA node.

[0107] The first NUMA node and the second NUMA node support multiple connection modes, and the first GPU and the second GPU support multiple paths. The connection modes of the first NUMA node and the second NUMA node on different paths are different.

[0108] The processor is used to schedule data transmission between the GPU on the first NUMA node and the GPU on the second NUMA node. The processor may be the first CPU in the first NUMA node, the second CPU on the second NUMA node, the CPU of other NUMA nodes other than the first NUMA node and the second NUMA node, or the CPU, MCU, etc. that do not belong to a NUMA node.

[0109] In one example, the electronic device does not include the first MUMA node and the second NUMA node. In this case, the processor is a CPU, an MCU, etc. that does not belong to a NUMA node.

[0110] In one example, the electronic device includes a first NUMA node, and the processor is a CPU, an MCU, etc. that does not belong to the NUMA node, or the processor is a first CPU.

[0111] In one example, the electronic device includes a second NUMA node, and the processor is a CPU, an MCU, etc. that does not belong to the NUMA node, or the processor is a second CPU.

[0112] In one example, the electronic device includes a first NUMA node and a second NUMA node, and the processor is a CPU, an MCU, etc. that does not belong to a NUMA node, or the processor is a first CPU, or the processor is a second CPU.

[0113] In one example, the electronic device includes a first NUMA node and a second NUMA node, and the first CPU is a CPU, MCU, etc. that does not belong to a NUMA node, or is a CPU in a NUMA node included in the electronic device. Here, the NUMA node where the processor is located may be the first NUMA node, the second NUMA node, or other NUMA nodes other than the first NUMA node and the second NUMA node.

[0114] When the processor determines that there is first data that needs to be transferred from the first GPU to the second GPU, the size of the data block included in the first data, i.e., the first data block size of the first data, is determined. The first data block size can be determined by the data block included in the first data, such as 1 megabyte (M), 2M, 3M, 5M, 10, etc.

[0115] The first data may be image data, model parameters, training data for model training, loss data for updating model parameters, and other data. The data processing method provided in the embodiment of the present application can be applied to various scenarios including but not limited to model training, data processing, image processing, and the like.

[0116] In an embodiment of the present application, the first data may include multiple first data blocks, and the sizes of the data blocks are the same or different. The size of the data block included in the first data may be strictly the first data block size or close to the first data block size. Here, the size of the data block close to the first data block size can be understood as the data difference between the size of the data block and the size of the first data block is less than a set data difference threshold, or the data difference between the size of the data block of the data block that exceeds a set proportion in the data blocks included in the first data and the size of the first data block is less than a set data difference threshold.

[0117] In an example, the first data includes multiple data blocks, and the data block size of each data block is 1M, then the first data block size of the first data is 1M.

[0118] In one example, the data difference threshold is 0.1M, and the data block sizes of the data blocks included in the first data are as follows: 0.98M, 0.99M, 1M, 0.99M, 1.01M, 1.05M, 1.1M, 1M, and 0.99M. The data block size of each data block has a data difference of less than 0.1M with respect to 1M, and it can be considered that the first data block size of the first data is 1M.

[0119] In one example, the data difference threshold is 0.1M, the set ratio is 0.9, and the data block sizes of the data blocks included in the first data are as follows: 0.98M, 0.99M, 1M, 0.6M, 1.01M, 1.05M, 1.1M, 1M, 0.99M, and 1.02M, among which, the data block sizes of 90% of the data blocks have a data difference of less than 0.1M with respect to 1M, and it can be considered that the first data block size of the first data is 1M.

[0120] In an embodiment of the present application, the data difference threshold may be a predefined fixed value, such as 0.1M, 0.5M, etc., or may be a value that changes according to the size of the first data block, for example: for 1M, the data difference threshold is 0.1M, for 2M, the data difference threshold is 0.2M, for 5M, the data difference threshold is 1M, and for 10M, the data difference threshold is 3M.

[0121] The set ratio can be a predefined fixed value, such as 5%, 10%, 20%, etc.

[0122] The processor determines transmission performance information of a first data block size, where the transmission performance information includes at least one of the following information: load information, delay information, throughput, etc., which can reflect the transmission performance when transmitting data blocks of the first data block size.

[0123] In the embodiment of the present application, the transmission performance information of the first data block size may be historical information collected by the processor when transmitting the first data block size.

[0124] In an embodiment of the present application, when data transmission of different data block sizes is performed, transmission performance information of the corresponding data block sizes is collected and recorded, so that when data transmission of a certain data block size is required, the transmission performance information of the data block size can be obtained from the recorded information.

[0125] In one example, the electronic device records transmission performance information of each data block size of 1M, 2M, 3M, 5M, and 10M. When the first data block size is 2M, the transmission performance information of 1M is obtained from the recorded transmission performance information.

[0126] In the embodiment of the present application, the transmission performance information of the first data block size may be information collected when the first data block size is transmitted based on a configured scheduling policy associated with the first data block size.

[0127] S202: The processor determines a scheduling strategy based on the transmission performance information of the first data block size, where the scheduling strategy is used to characterize a transmission method for transmitting the first data through the at least two paths.

[0128] The processor determines a scheduling strategy based on the transmission performance information of the first data block size. Here, the determined scheduling strategy is used for how to transmit the first data on multiple paths supported by the first path when the first data is transmitted on the first path, such as: which paths are used for transmission, the data distribution ratio of each path among the multiple paths supported by the first path, etc.

[0129] S203: The processor schedules, based on the scheduling policy, the first data to be transmitted from the first GPU to the second GPU through the at least two paths.

[0130] After determining the scheduling strategy, the processor controls the first data to be transmitted from the first GPU to the second GPU through the multiple paths supported by the first path based on the scheduling strategy.

[0131] In a possible implementation, the electronic device where the processor is located does not include the first NUMA node and the second NUMA node, that is, the processor and the first NUMA node and the second NUMA node are located in different electronic devices, then the processor sends the scheduling policy to the electronic device where the first NUMA node and the second NUMA node are located through the connected transceiver, so that the electronic device where the first NUMA node and the second NUMA node are located sends the first data from the first GPU to the second GPU based on the scheduling policy. Among them, for the electronic device where the first NUMA node and the second NUMA node are located, a first thread can be created and run, and the first thread transmits the first data from the first GPU to the second GPU based on the scheduling policy. The first thread is a thread that transmits the first data. In one example, the first thread is a global reduce (allreduce) thread.

[0132] In a possible implementation, the processor and the first NUMA node and the second NUMA node are located in the same electronic device, and the processor creates and runs a first thread, where the first thread is a thread for transmitting the first data. The first thread accesses the operation entrances of each path supported by the first path, reads the data transmitted (responsible for transmission) by each path in the first data, and transmits the data transmitted by each path through the corresponding paths from the first GPU to the second GPU.

[0133] In the embodiment of the present application, the operation entry may also be referred to as the operation function entry.

[0134] In the electronic device and data processing method provided by the embodiments of the present application, a first path between GPUs supports multiple channels, and when there is first data to be transmitted on the path, transmission performance information is determined based on the data block size of the first data, and a scheduling strategy for transmitting the first data on the multiple channels supported by the first path is determined based on the transmission performance information, and the first data is scheduled to be transmitted from one GPU to another GPU through the multiple channels supported by the first path based on the scheduling strategy; here, the scheduling strategy for transmitting the first data on multiple channels is determined based on the transmission performance information of the current data block size of the first data, and the transmission mode on the multiple channels can be effectively controlled according to the transmission performance of the current data block size, and the transmission of the first data can be reasonably scheduled, the transmission capacity of various channels is balanced, and the overall throughput capacity of the first path is improved.

[0135] In a possible implementation, S202 determines a scheduling strategy based on the transmission performance information of the first data block size, including: selecting a candidate scheduling strategy corresponding to the transmission performance information as a scheduling strategy from multiple candidate scheduling strategies.

[0136] Here, multiple candidate scheduling strategies may be set, and different candidate scheduling strategies correspond to different transmission performance levels, and the candidate scheduling strategy corresponding to the transmission performance level of the transmission performance information of the first data transmission block of the first data is determined as the scheduling strategy. Wherein, when the transmission performance information includes one information, the transmission performance level of the information is determined, and when the transmission performance information includes multiple information, the multiple information may be weighted to obtain a transmission performance score, and the transmission performance level corresponding to the transmission performance score is determined.

[0137] In a possible implementation, S202 determines the scheduling strategy based on the transmission performance information of the first data block size, including: obtaining a configured scheduling strategy associated with the first data block size; and adjusting the configured scheduling strategy according to the transmission performance information to obtain the scheduling strategy.

[0138] In the embodiment of the present application, the configured scheduling strategy can be the scheduling strategy adopted for the data transmission for which the collected transmission performance information is directed. It is understandable that the configured scheduling strategy is adopted to transmit data, and the data block size of the transmitted data is the first data block size, and the transmission process of the data is monitored to obtain the transmission performance information.

[0139] When the transmission performance information indicates that the data transmission performance is good, the configured scheduling policy may be modified with a small floating amount. When the transmission performance information indicates that the data transmission performance is poor, the configured scheduling policy may be modified with a large floating amount.

[0140] Taking the transmission performance strategy including the allocation ratio of each of the multiple paths as an example, the allocation ratio of each of the multiple paths included in the configured scheduling strategy is adjusted to obtain the scheduling strategy.

[0141] In the embodiment of the present application, the allocation ratio of each channel may be adjusted based on the performance of each channel for transmission of the first data block size.

[0142] In one example, the first path supports path 1 and path 2. Compared with small data blocks, path 1 has better transmission performance when transmitting large data blocks, and compared with large data blocks, path 2 has better transmission performance when transmitting small data blocks. If the first data block is a large data block, the allocation ratio corresponding to path 1 can be increased and / or the allocation ratio corresponding to path 2 can be decreased based on the configured scheduling strategy to obtain the scheduling strategy. Whether the data block belongs to a large data block or a small data block can be determined based on a set data block size threshold.

[0143] In one example, the first path supports channel 1 and channel 2. The allocation ratios of channel 1 and channel 2 in the configured scheduling strategy are 30% and 70% respectively. The data allocation ratio of each channel in the configured scheduling strategy is adjusted through transmission performance information. In the resulting scheduling strategy, the allocation ratios of channel 1 and channel 2 are 40% and 60% respectively.

[0144] In the embodiment of the present application, for the allocation ratio that needs to be adjusted, the adjustment step size of each path can be set, and the adjustment amount of each path can be determined according to the transmission performance information. The adjustment degree of the path is determined based on the adjustment step size and adjustment amount of each path. For a path, the allocation ratio of the path in the configured scheduling strategy is adjusted based on the adjustment degree of the path.

[0145] In the embodiment of the present application, the scheduling strategy can be obtained by adjusting the configured scheduling strategy associated with the first data block size, so as to obtain a scheduling strategy with better transmission performance as soon as possible.

[0146] In some embodiments, the transmission performance information includes first load information based on Figure 2 , the method further comprises Figure 3 Steps shown:

[0147] S301, obtaining a first number of second load information of a first thread, where the first thread is a thread that transmits the first data, and the second load information is load information of the first thread when transmitting a data block of the first data block size;

[0148] S302: Determine the first load information based on the first number of the second load information.

[0149] The first quantity second load information of the first thread obtained by the processor can be collected by the processor when the processor and the first NUMA node and the second NUMA node are located in the same electronic device, or can be received by the processor from the electronic device where the first NUMA node and the second NUMA node are located when the processor and the first NUMA node and the second NUMA node are located in different electronic devices.

[0150] When the processor, the first NUMA node and the second NUMA node are located in the same electronic device, a timer may be provided in the electronic device, and the timer may record load information during each data transmission. The processor obtains the most recent first number of load information recorded by the timer, that is, the first number of second load information.

[0151] The processor may process a first number of second load information to obtain first load information.

[0152] In one example, the first load information is an average value of a first number of second load information.

[0153] In one example, the first load information is a median of a first number of second load information.

[0154] In one example, the first load information is the mode of a first number of second load information.

[0155] In the embodiment of the present application, there is no limitation on the method of determining the first load information based on the first amount of second load information.

[0156] In an embodiment of the present application, the load overhead of the first data block size can be recorded, and a scheduling strategy can be determined based on the load overhead of the first data block size over a period of time, thereby ensuring that the loads of multiple paths of the first path are controlled in a balanced state, thereby improving parallelism and bandwidth utilization.

[0157] In some embodiments, the at least two pathways include at least two of the following pathways:

[0158] A first path, in which the first GPU is connected to the second GPU via a bounce buffer;

[0159] a second path, in which the first GPU is connected to the second GPU via a non-transparent bridge NTB;

[0160] A third path, in which the first CPU in the first NUMA node is connected to the second CPU in the second NUMA node through a quick path interconnect (QPI).

[0161] The scheduling policy determined by the processor based on the data transmission performance information may be used to schedule transmission of the first data on at least two of the first path, the second path, and the third path.

[0162] In the embodiment of the present application, the first path may support at least two of the first path, the second path and the third path.

[0163] In one example, if Figure 4A As shown, the path between the first GPU 401 and the second GPU 402 includes a first path 403 , a second path 404 , and a third path 405 .

[0164] In one example, if Figure 4B As shown, the path between the first GPU 401 and the second GPU 402 includes a first path 403 and a second path 404 .

[0165] In one example, if Figure 4C As shown, the path between the first GPU 401 and the second GPU 402 includes a second path 404 and a third path 405 .

[0166] For the first path, the first NUMA node and the second NUMA node are connected through a bounce buffer, so that the first GPU and the second GPU perform data transmission through the bounce buffer. The first path can also be called a bounce buffer path or a bounce buffer data channel. The bounce buffer can also be called a page-locked memory.

[0167] When the first GPU transmits data to the second GPU through the first path, a bouncebuffer memory block can be created in the second CPU memory, the data of the first GPU can be copied to the bouncebuffer memory block, and then the second GPU reads the data from the bouncebuffer memory block, thereby realizing indirect access of the second GPU to the data of the first GPU. The first GPU and the second GPU can both access the bouncebuffer memory block, so the bouncebuffer memory block can be understood as a shared memory.

[0168] In the embodiment of the present application, the first path will apply for a relatively large memory pool for communication because it is transmitted through shared memory, so the larger the data packet, the better the performance. For large data blocks, priority can be given to the first path. In one example, when transmitting large data blocks larger than 2M, in the scheduling strategy, a relatively large allocation ratio (proportion) can be scheduled to the first path, thereby improving the overall data transmission performance.

[0169] For the second path, the first NUMA node and the second NUMA node are connected through a non-transparent bridge (Non-Transparent Bridging, NTB), so that the first GPU and the second GPU perform data transmission through the NTB. The second path can also be called an NTB path or an NTB data channel.

[0170] When the first GPU transmits data to the second GPU through the second path, an NTB memory block can be created on the NTB, the data of the first GPU can be copied to the NTB memory block, and then the second GPU reads the data from the NTB memory block, thereby realizing indirect access of the second GPU to the data of the first GPU.

[0171] Since NTB hardware has high requirements for timing, the second path is more sensitive to latency, so the second path is faster for processing small data packets. For small data blocks, they can be scheduled to the second path first. In one example, for data blocks smaller than 2M, a larger proportion can be scheduled to the second path in the scheduling strategy.

[0172] For the third path, the first NUMA node and the second NUMA node are connected via a Quick Path Interconnect (QPI), so that the first GPU and the second GPU perform data transmission via the QPI. The third path may also be referred to as a QPI path or a QPI data path.

[0173] In an example, the first CPU and the second CPU may be connected through a QPI, so that the first NUMA node and the second NUMA node are connected through the QPI.

[0174] It should be noted that when the first path or the second path performs data transmission, the data transmission process may not pass through the first CPU and the second CPU.

[0175] In the embodiment of the present application, the multiple paths supported by the first path may respectively be paths with different transmission performances on different data blocks, thereby improving the overall throughput while taking into account data transmission scenarios with different data block sizes.

[0176] In some embodiments, the scheduling strategy includes a data allocation ratio of each of the at least two paths; S203, based on the scheduling strategy, scheduling the first data to be transmitted from the first GPU to the second GPU through the at least two paths, includes:

[0177] S2031, determining at least one target path among the at least two paths according to the data allocation ratio of each path among the at least two paths, and determining an access address corresponding to each target path among the at least one target path according to the data allocation ratio of each target path among the at least one target path and a storage location of the first data in the memory of the first GPU, wherein the target path is a path used for transmission of the first data;

[0178] S2032. For each target path in the at least one target path, schedule second data to be transmitted from the first GPU to the second GPU through the target path based on an access address corresponding to the target path, where the second data is data in the first data transmitted using the target path.

[0179] In the embodiment of the present application, the processor may determine the target path for the transmission of the first data among the at least two paths supported by the first path according to the scheduling strategy, wherein the target path is a path whose corresponding allocation ratio is not zero. If the data allocation ratio of a path in the scheduling strategy is zero, it means that the path is not used for the transmission of the first data, and if the data allocation ratio of a path in the scheduling strategy is not zero, it means that the path is used for the transmission of the first data, that is, the path is the target path.

[0180] In one example, the first path supports channel 1, channel 2, and channel 3. In the scheduling strategy, the corresponding allocation ratios of channel 1, channel 2, and channel 3 are 40%, 60%, and 0, respectively. Then the target channels include channel 1 and channel 2.

[0181] The first data is stored in the memory of the first GPU, and the processor can determine the access address of each target path to the memory of the first GPU according to the allocation ratio of each target path in the scheduling strategy, wherein the access address corresponding to each target path stores the second data for transmission of the target path.

[0182] In one example, the target path includes: path 1 and path 2, the allocations corresponding to path 1 and path 2 in the scheduling strategy are 20% and 80% respectively, the storage location of the first data in the first GPU memory is the storage area from address 1 to address 2, then based on the storage area of ​​address 1 and address 2 and 20% and 30%, it is determined that the access addresses of path 1 and path 2 are address 1 and address 3 respectively, wherein the area from address 1 to address 3 accounts for 20% of the storage area of ​​address 1 and address 2, and the area from address 3 to address 2 accounts for 80% of the storage area of ​​address 1 and address 2.

[0183] When the processor and the first NUMA node and the second NUMA node are located in the same electronic device, after determining the access address corresponding to each target path, the processor points the operation entry of each path to the corresponding access address, and the first thread accesses the access address of the memory of the first GPU based on each target path, thereby reading the second data for transmission of each target path.

[0184] In the case where the processor and the first NUMA node and the second NUMA node are located in different electronic devices, the processor determines the access address corresponding to each target path, and then sends the access address corresponding to each target path to the electronic device where the first NUMA node and the second NUMA node are located. The operation entry of each path in the electronic device is directed to the corresponding access address, and the first thread accesses the access address of the memory of the first GPU based on each target path, thereby reading the second data for transmission of each target path.

[0185] In a possible implementation, S2031 determines, according to the data allocation ratio of each target path in the at least one target path and the storage location of the first data in the memory of the first GPU, an access address corresponding to each target path in the at least one target path, including:

[0186] S2311, determining a data size of the second data used for transmission by each target path in the at least one target path in the first data according to the size of the first data and the data allocation ratio of each target path in the at least one target path;

[0187] S2312. For each target path in the at least one target path, determine an access address corresponding to the target path according to a data size of second data used to transmit the target path and a first address of the first data, where the first address is a starting address of a storage location of the first data in the memory of the first GPU.

[0188] The size of the space occupied by the first data in the memory of the first GPU is the size of the first data. In one example, if the first data is 100M, then the first data occupies 100M in the memory of the first GPU. In another example, if the first data is 30M, then the first data occupies 30M in the memory of the first GPU.

[0189] The processor determines the size of the second data that each target channel is responsible for transmitting according to the size of the first data and the data allocation ratio of each target channel in the scheduling strategy.

[0190] In one example, the size of the first data is 100M, and the target paths include: path 1 and path 2. In the scheduling strategy, the allocation ratios of path 1 and path 2 are 40% and 60% respectively. Then, path 1 is used to transmit 40M of the first data, that is, the size of the second data transmitted by path 1 is 40M, and path 2 is used to transmit 60M of the first data, that is, the size of the second data transmitted by path 2 is 60M.

[0191] The processor determines the access address corresponding to each path according to the size of the second data used for transmission by each path and the starting address of the storage location of the first data in the first GPU.

[0192] In one example, the starting address of the storage location of the first data in the first GPU is: address 1, the size of the second data used to transmit by path 1 is 40M, and the size of the second data used to transmit by path 2 is 60M, then the access address of path 1 is address 1, and the access address of path 2 is address 2, wherein the size of the space between address 1 and address 2 is 40M.

[0193] In the embodiment of the present application, the access address of each channel is accurately determined based on the size of the first data and the position of the first data in the first GPU memory, so as to accurately read the second data for transmission of each channel.

[0194] In some embodiments, the electronic device includes the second NUMA node, and S2032 schedules the second data to be transmitted from the first GPU to the second GPU through the target path based on the access address corresponding to the target path, including:

[0195] S2321, read the second data from the access address corresponding to the target path, and copy the read second data to a shared memory area, where the shared memory area is a mapping of a bounce buffer or an NTB in a CPU memory of a second NUMA node;

[0196] S2322, scheduling the second GPU to read the second data from the shared memory area; scheduling the second GPU to copy the second data to the memory of the second GPU.

[0197] Here, based on Figure 1 The electronic equipment shown, such as Figure 5 As shown, the electronic device 100 includes a processor 101, a first GPU 102, and a second GPU 103, wherein the processor 101 and the first GPU 102 belong to a first NUMA node, or the processor 101 and the second GPU 103 belong to a second NUMA node, or the processor 101 does not belong to the first NUMA node nor to the second NUMA node.

[0198] based on Figure 5 The electronic device shown in FIG. 1 is used to transmit the second data on the target path. Figure 6 As shown, including:

[0199] S601, copying the second data in the first GPU memory to the shared memory area in the second NUMA node;

[0200] S602: The second GPU reads second data from the shared memory area;

[0201] S603: The second FPU copies the read second data to the second GPU memory.

[0202] In the case where the first NUMA node and the second NUMA node are located in the same electronic device as the processor, the processor sends the access address of each target path to the operation entry of the corresponding target path. For a target path, the first thread accesses the operation entry of the target path to read the second data for transmission of the target path from the first GPU memory, and copies the read second data to the shared memory area.

[0203] The shared memory area is an area that can be accessed by both the first GPU and the second GPU. The shared memory area in the embodiment of the present application may be located in the CPU memory area in the second NUMA node, which is a partial area in the CPU memory area in the second NUMA node.

[0204] For the first path, the shared memory area is the mapping of the bouncebuffer in the CPU memory of the first NUMA node or the mapping in the CPU memory of the second NUMA node, which can be understood as the area in the CPU memory of the first NUMA node or the second NUMA node mapped to the bouncebuffer. For the second path, the mapping of the shared memory area NTB in the CPU memory of the first NUMA node or the mapping in the CPU memory of the second NUMA node can be understood as the area in the CPU memory of the first NUMA node or the second NUMA node mapped to NTB. Among them, the shared memory area corresponding to the first path and the shared memory area corresponding to the second path are different areas in the CPU memory.

[0205] For a target path, after the second data is copied from the memory of the first GPU to the shared memory area, the first thread schedules the second GPU to read the second data from the shared memory area, and schedules the second GPU to copy the read second data to the memory of the second GPU, thereby realizing the transmission of the second data from the first GPU to the second GPU through the target path.

[0206] It can be understood that when the second data is transmitted from the first GPU to the second GPU through the target path, the first data blocks included in the second data are transmitted one by one from the first GPU to the second GPU through the target path with the first data blocks as the granularity.

[0207] In the embodiment of the present application, the first thread may directly schedule the second GPU to read the second data from the shared memory area, or may schedule the second GPU to read the second data from the shared memory area when determining that the second data in the shared memory area is available.

[0208] Taking the example that the first thread can schedule the second GPU to read the second data from the shared memory area, the first thread can schedule the second GPU to read the first data block included in the second data from the shared memory area.

[0209] Taking the example of scheduling the second GPU to read the second data from the shared memory area when the first thread determines that the second data in the shared memory area is available, when the first thread determines that all the second data in the shared memory area are copied to the target path for transmission, the second GPU can be scheduled to read the first data block included in the second data from the shared memory area.

[0210] In a possible implementation, the second GPU periodically reads data in the shared memory area, and when storing the second data in the shared memory area, the second GPU reads the second data in the shared memory area.

[0211] In a possible implementation, scheduling the second GPU to read the second data from the shared memory area in S2322 includes: scheduling the bounce buffer to send a notification to the second GPU when the second data in the first shared memory area is available, the notification being used to instruct the second GPU to read the second data; scheduling the second GPU to read the second data from the shared memory area in response to the notification.

[0212] At this time, the second GPU triggers the reading of the second data based on the received notification. For the first path, the notification may be sent to the second GPU by the bouncebuffer. For the second path, the notification may be sent to the second GPU by the NTB.

[0213] In some embodiments, the method further comprises:

[0214] Before reading the second data from the access address corresponding to the target path at S2321, the processor initializes the shared memory in the CPU memory and creates a mapping of the bounce buffer or the NTB to the shared memory area in the shared memory;

[0215] After scheduling the second GPU to copy the second data to the memory of the second GPU at S2322, the processor releases the mapping of the bounce buffer or the NTB to the shared memory area.

[0216] In the embodiment of the present application, the shared memory may be an area divided out from the CPU memory area of ​​the second NUMA for the first GPU and the second GPU to access together.

[0217] The processor creates a mapping from the bounce buffer or the NTB to the shared memory area in the shared memory, that is, determines the shared memory area, so as to cache the second data of the corresponding path when using the first path or the second path for data transmission.

[0218] After the transmission of the second data from the first GPU to the second GPU is completed through the target path, the bounce buffer or the mapping of the NTB to the shared memory area is released in the shared memory area, thereby releasing the shared memory area.

[0219] In the embodiment of the present application, when data needs to be transmitted through the target path, a shared memory area is created for the target path, and when data transmission is completed through the target path, the shared memory area is released, thereby achieving effective utilization of memory resources.

[0220] In some embodiments, the electronic device includes the first NUMA node and the second NUMA node, and the same model is running in the first GPU and the second GPU, and the method also includes: the second GPU processes the first data based on the running model to obtain a first output result; the first GPU processes the third data based on the running model to obtain a second output result; the processor schedules the first data output result to be transmitted to the first GPU; the first GPU obtains fourth data based on the first output result and the second output result, and the fourth data is used to determine the loss or to update the parameters of the model.

[0221] In the embodiment of the present application, multiple GPUs can be used for parallel training of the model, that is, to implement multi-GPU model training. Here, the first data is the distributed training data, thereby implementing model training.

[0222] In a model training scenario, the same model can be run on multiple GPUs, and the training data input to the model is divided into multiple parts, where the first data and the third data are two of the multiple parts, the input of the first GPU is the third data, and the input of the second GPU is the first data.

[0223] Here, if Figure 7As shown, the first GPU 701 can divide the training data into multiple parts, and transmit the first data to the second GPU 702. The model on the second GPU 702 outputs the first output result when the first data is input, and the model on the first GPU 701 outputs the second output result when the third data is input, and the second GPU 702 transmits the second output result to the first GPU 701, and the first GPU 701 aggregates the output results of multiple GPUs to obtain an aggregated output result, and the aggregated output result is used to determine the total loss. This process can be understood as the forward propagation process of model training.

[0224] In one possible implementation, the first GPU updates the model parameters based on the total loss to complete one iteration of training.

[0225] In one possible implementation, the first GPU determines the gradient of the loss based on the total loss, and distributes the calculated loss gradient to all GPUs. Each GPU calculates the gradients of all model parameters based on the received loss gradient. All GPUs aggregate the calculated parameter gradients to the first GPU. The first GPU updates the model parameters based on the aggregated gradients to complete one iteration of training.

[0226] At this time, the first GPU transmits the first loss gradient corresponding to the second GPU to the second GPU, the second GPU calculates the first parameter gradient based on the first loss gradient, the first GPU calculates the second parameter gradient based on its own second loss gradient, the second GPU transmits the first parameter gradient to the first GPU, and the first GPU updates the model parameters based on the summarized parameter gradient. This process can be understood as the back propagation process of model training.

[0227] In the embodiment of the present application, running the same model on multiple GPUs is a case of data parallelism. In practical applications, the data processing method provided in the embodiment of the present application can be applied to the case of model parallelism. At this time, parts of the model are run on multiple GPUs respectively, thereby realizing model training. Here, the case of model parallelism is not described in detail.

[0228] It should be noted that the manner in which the second GPU transmits data to the first GPU may refer to the manner in which the first GPU transmits data to the second GPU, which will not be described in detail here.

[0229] The following describes the data processing method provided in the embodiments of the present application.

[0230] The number of parameters in deep learning (DL) models has exploded, and a single graphics processing unit (GPU) can no longer complete the training task. In order to train such models, parallel training distributes computing and communication requirements to multiple GPUs, and uses communication between GPUs to share computing results or synchronize training status. Therefore, the performance of GPU communication becomes critical to training throughput. GPU manufacturers provide specialized communication libraries for GPU products and use a variety of technologies to optimize communication.

[0231] In the related technology, GPU communication technology includes high-speed interconnection technology NVLink interconnection technology and high-speed serial computer expansion bus standard (peripheral component interconnect express, PCIe) interconnection technology.

[0232] GPU interconnection technology uses hardware connection technologies such as NVLink or NVSwitch to achieve full-link connection of GPUs and has good performance. However, GPU manufacturers set technical thresholds and the application system must be highly dependent on NVIDIA's GPU hardware.

[0233] PCIe interconnect technology is a system connected by the PCIe bus within the system. The GPUs of this system must communicate through the PCIe host bridge and the QPI bus interconnection between the NUMA architecture. According to the analysis of the GPU cluster trace, the system that relies on PCIe covers 90.4% of the machine specifications that appear in the trace.

[0234] In different NUMA nodes, QPI or Infinity Fabric technology is used to connect different nodes to enable communication between GPUs, CPUs, and memories of different nodes. However, the speed at which the GPUs and CPUs of NUMA nodes access the memory of remote NUMA nodes is slower than the speed at which they access the memory of GPUs and CPUs within a NUMA node.

[0235] like Figure 8 As shown, in the hardware environment, four V100 GPUs are installed on the dual numa processors: G0, G1, G2, and G3. Among them, G0 and G1 of the numa0 node communicate with G2 of the numa1 node. The transmission topology between the numa0 node and the numa1 node may include the topology shown in Table 1. Among them, in the topology shown in Table 1, the first group describes that the access rate between GPUs in different NUMA nodes is congested. Fig. 9As shown in the figure, the performance is reduced because the two overlap on the PCIe link between G1 and CPU when transmitting at the same time. However, through analysis, the link is bidirectional and the link bandwidth is sufficient. If the congestion of this link can be avoided, theoretically they should achieve better performance.

[0236] Table 1. Example of transmission paths on dual NUMA processors

[0237] No. Hardware Type Transmission topology Bandwidth (GB / s) 1 V100 {G0->G1, G1->G2} 4.42 2 V100 {G0->G2} 9.45 3 V100 {G0->M0} 13.16 4 V100 {G0->M0,M0->G1, G1->M1,M1->G2} 12.29

[0238] As shown in the second and third groups of transmission paths, if each group is executed separately, GPU0 can get a better bandwidth of 9.45 GB / s when writing to the memory of another GPU2 in the remote NUMA node. When GPU0 writes to the CPU memory M0 inside the local NUMA node, it can also achieve 13.16 GB / s.

[0239] Transmission is facilitated by BounceBuffer in CPU memory. For example, for the fourth group of transmission paths in Table 1, BounceBuffer is dynamically allocated on different numa nodes in combination with the different numa nodes where the GPU is located, the virtual topology of the communication algorithm is changed, and BounceBuffer is used to form a network between GPUs. Fig.10 The new ring topology (Ring-Topo) shown can reduce congestion and make full use of better paths to enhance communication capabilities.

[0240] The data processing method provided in the embodiment of the present application involves the following contents:

[0241] Content 1: Combining GPUI and NUMA node distribution, adaptively allocating BounceBuffer to different NUMA nodes, and comprehensively improving the computing and communication performance of the cluster by changing the virtual topology order between GPUs in the collective communication Ring algorithm;

[0242] Content 2: Based on the above mechanism that can flexibly change the virtual topology, by establishing the mapping translation of the base address register, we can connect the faster and more stable NTB non-transparent bridge hardware in series, making the overall path more diverse;

[0243] Content 3: Non-transparent bridge communication mode is a communication mode used to connect PCIe devices and peripherals. In NTB mode, PCIe devices and peripherals can communicate directly without CPU intervention. This can significantly improve communication efficiency and reduce latency.

[0244] In the data processing method of the embodiment of the present application, the following preparations may be performed before data transmission:

[0245] Building a connection topology

[0246] The objects of a heterogeneous computing cluster include the CPU, GPU, memory and other devices installed on the server, as well as the link facilities connecting these devices. The physical connection relationship between the devices is constructed as follows: Fig.11 As shown, multiple GPUs are connected together through a PCIe bridge and then connected in series based on a PCIe bus. For devices across NUMA, a ring connection topology (path) is formed through a QPI bus in series.

[0247] In order to maximize the performance of GPU and other heterogeneous hardware, such as Fig.11 As shown, a method for creating a BounceBuffer memory block is also proposed. Through address mapping, the virtual topology order between GPUs is changed, and cross-NUMA GPU devices are connected in series through BounceBuffer to form another ring topology.

[0248] Based on the above, a mechanism for flexibly changing the virtual topology is opened up.

[0249] like Fig.11 As shown, a method of connecting NTBs between NUMA nodes is also proposed. By establishing a mapping translation of the base address register, the faster and more stable NTB non-transparent bridge hardware is connected in series, making the overall path more diverse.

[0250] 2. Build a communication channel

[0251] In the present application embodiment, Fig.12 As shown in the figure, based on three different topologies of QPI, BounceBuffer and NTB, three communication channels are constructed respectively: QPI Channel, BounceBuffer Channel and NTB Channel. For the convenience of management, the communication channel is abstracted as Channel at the software level, and at the hardware level, it corresponds to a group of hardware stream processing threads to perform communication and computing tasks in parallel.

[0252] 3. Communication channel strategy

[0253] The communication channel strategy can be customized, in which you can configure which link (channel) to use and the data allocation ratio of each channel. Since the bandwidths of the three different channels are different, data can be divided into different channels according to a certain ratio to ensure that their loads are controlled in a balanced state, thereby improving parallelism and bandwidth utilization.

[0254] In the embodiments of the present application, 1) the virtual topology of the collective communication algorithm is changed. For example, when the Ring algorithm is used, the order of GPUs in the Ring can be changed to change the GPU communication path to the maximum extent to avoid communication congestion; 2) when transmitting communications between GPUs, combined with the characteristics of the NUMA architecture, the BounceBuffer or NTB link on the CPU memory is adaptively utilized to improve the overall bandwidth capability.

[0255] The data processing method provided in the embodiment of the present application has 1) strong scalability and high performance, and optimizes the virtual topology based on the Ring-Topo algorithm, which fully utilizes the existing capabilities and improves the bandwidth capacity; 2) does not require application adaptation and modification, and has low development and operation costs.

[0256] The data processing method provided in the embodiment of the present application includes the following processes: multi-path replication and multi-path channel allocation.

[0257] Multipath replication process

[0258] The GPU-to-GPU point-to-point multipath data transmission mechanism can divide point-to-point messages across device-to-device (D2D) and device-to-host (D2H) / host-to-device (H2D) channels to utilize all available transmission paths, while dynamically selecting NTB or BounceBuffer channels at different stages to maximize the overlap of communication and computation and improve overall throughput.

[0259] Sending data between GPUs through the host requires temporary storage in the host memory. Split data transferred to the host memory into multiple data blocks, which are placed on individual streams to allow pipelining between D2H and H2D replicas. Send approximately 5-10% of the messages through the host, and the rest directly to adjacent GPUs. Send different amounts of data for each path to account for bandwidth mismatches. Record the optimal number of blocks and message distribution for each message size in a static adjustment table.

[0260] like Fig.13 As shown, the exemplary process of using the BounceBuffer channel includes:

[0261] S1301, the CPU acts as a controller and is responsible for initialization operations, initializing GPU memory and Bouncebuffer memory, and mapping the address space of BounceBuffer to a common memory space, so that GPUs across NUMA can access each other's address space.

[0262] S1302, determine the data size that GPU4 sends through the host or directly to another GPU: GPU5. Once the data size to be sent is determined, copy the data block to the shared memory area mapped by bouncebuffer in the host memory.

[0263] S1303, when the data block is available in the host memory, notify the target GPU: GPU5 reads the data on the shared memory area;

[0264] S1304 , read data from the shared memory area mapped by bouncebuffer, and copy it to the target GPU.

[0265] S1305: Release the BounceBuffer memory mapping, and the GPU releases resources.

[0266] like Fig.14 As shown, an exemplary process of using the NTB channel includes:

[0267] S1401, the CPU acts as a controller and is responsible for initialization operations, initializing GPU memory and NTB memory, and mapping the address space of the NTB to a common memory space, so that GPUs across NUMA can access each other's address space.

[0268] S1402, determine the data size that GPU4 sends through the host or directly to another GPU: GPU5. Once the data size to be sent is determined, copy the data block to the shared memory area mapped by NTB in the host memory.

[0269] S1403, when the data block is available in the host memory, notify the target GPU: GPU5 reads the data on the shared memory area;

[0270] S1404 , read data from the shared memory area mapped by the NTB, and copy it to the target GPU.

[0271] S1405: Unmap the NTB memory, and the GPU releases resources.

[0272] Multipath channel allocation

[0273] When the system is training, it can obtain the performance indicators of the current channel in real time, such as the throughput and latency when a round of data is completed, dynamically monitor the load of all channels, and reasonably allocate data flow to avoid overloading a certain channel.

[0274] Especially in the process of model training, many parameters are applied. In the collective communication module, data of different packet lengths are reflected. In end-to-end training, small data packets less than 2M frequently occupy GPU system resources, which will cause great pressure on the system. Since the BounceBuffer data channel transmits through shared memory, it will apply for a relatively large memory pool for communication. Therefore, the performance is better for the larger the data packet, which can be used to schedule a large proportion of data larger than 2M to the BounceBuffer channel; NTB hardware has high requirements for timing, and the NTB data channel is more sensitive to latency, so the NTB channel processes small data packets faster, and data packets less than 2M can be scheduled to the NTB data channel first.

[0275] The operation entry (Opts) of each channel obtains the memory address of the data through a pointer. Then the allreduce thread accesses Opts, reads the data it is responsible for, and initiates data transmission. When the communication starts, the Timer records the load of each operation according to the thread ID. In order to reduce decision errors caused by fluctuations, the Timer transmits the average load indicator of every 10 allreduce operations of the same data size to the analyzer. The analyzer adjusts the data volume of each channel and the corresponding data acquisition pointer. Different channels will exhibit different latency performance when processing different data sizes. Therefore, the analyzer maintains a data length table to record the proportion of data that each channel member should be responsible for when processing different data sizes.

[0276] Exception detection stores the result data as UnboundBuffer. If the current operation does not return complete result data, the completed network wait is requested to ensure the correctness of data transmission. If a network failure occurs, the exception handler records the error network object, providing error details such as memory access violation or address conflict.

[0277] The data processing method of the embodiment of the present application includes the following steps: initializing a context object, including a thread and a memory address; a thread retrieving a data address; a timer recording the load overhead of a working thread; calculating a data partition ratio based on the load overhead; updating a data length and an operation pointer based on the partition ratio; and scheduling data to different channels based on the data length and the operation pointer.

[0278] In one example, if Fig.15As shown, the load overhead of the load collected by the timer includes: 100 microseconds (μs) for the NTB channel, 300μs for the bouncebuffer channel, and the records in the data length table include: for 1M, the scheduling strategy is that the allocation ratio of Opts1 is 30% and the allocation ratio of Opts2 is 70%, for 2M, the scheduling strategy is that the allocation ratio of Opts1 is 20% and the allocation ratio of Opts2 is 80%, the pointer Ptr1 of Opts1 points to the address 0x000351E, the pointer Ptr2 of Opts2 points to the address Ptr1+data length 1, and the data length 1 is the length of the data that Opts1 is responsible for transmitting. Among them, Opts1 is the operation entry of the NTB channel, and Opts2 is the operation entry of the bouncebuffer channel. When the data block size is 1M, the allocation ratio of Opts1 and Opts2 is adjusted for 1M, that is, the configured scheduling strategy. The adjusted scheduling strategy is: the allocation ratio of Opts1 is 33% and the allocation ratio of Opts2 is 67%. According to the modified adjustment strategy, the address of Ptr1 of Opts1 is determined to be 0x00000000 and the address of Ptr1 of Opts1 is 0x00080000, and data is accessed based on the pointers of each channel.

[0279] In one example, the channel policy configuration information used to configure the channel policy is as follows:

[0280] <transfer type="WRITE_GPUMEM" src="7" dst="6" / >

[0281] <transfer type="WRITE_GPUMEM"src="6"dst="5" / >

[0282] <transfer type="WRITE_GPUMEM"src="5"dst="4" / >

[0283] <transfer type="WRITE_CPUMEM"src="4"dst="1" / >

[0284] <transfer type="READ_CPUMEM"src="1"dst="3" / >

[0285] <transfer type="WRITE_GPUMEM"src="3"dst="2" / >

[0286] <transfer type="WRITE_GPUMEM"src="2"dst="1" / >

[0287] <transfer type="WRITE_GPUMEM"src="1"dst="0" / >

[0288] <transfer type="WRITE_CPUMEM"src="0"dst="0" / >

[0289] <transfer type="READ_CPUMEM"src="0"dst="7" / > .

[0290] The above channel strategy configuration represents the Ring topology of a single node (8 GPUs) (indexed from 0 to 7). Among them, src represents the previous GPU of the Ring, dst represents the next GPU of the Ring; CPUMEM represents the cross-NUMA stage of the path, which can be carried by NTB or BounceBuffer, and GPUMEM represents the GPU stage of the path.

[0291] The channel strategy configuration may also include the following information: indicating the type of channel path to be used, and indicating the measured bandwidth in Gbps.

[0292] In channel policy configuration,<transfer type="WRITE_GPUMEM"src="7"dst="6" / > ,<transfer type="WRITE_GPUMEM"src="6"dst="5" / > ,<transfer type="WRITE_GPUMEM"src="5"dst="4" / > "Indicates that all GPUs distributed under numa1 are interconnected using P2P;"<transfertype="WRITE_CPUMEM"src="4"dst="1" / > ,<transfer type="READ_CPUMEM"src="1"dst="3" / > "Indicates that GPU4 of numa1 is connected to GPU3 of numa0 through BouceBuffer of numa1;"<transfer type="WRITE_GPUMEM"src="3"dst="2" / > ,<transfer type="WRITE_GPUMEM"src="2"dst="1" / > ,<transfer type="WRITE_GPUMEM"src="1"dst="0" / > "Indicates that all GPUs in numa0 are interconnected via P2P;"<transfer type="WRITE_CPUMEM"src="0"dst="0" / > ,<transfer type="READ_CPUMEM"src="0"dst="7" / > " indicates that GPU0 of numa0 is connected to GPU7 of numa1 through BouceBuffer of numa0 to build an optimal ring path channel.

[0293] In the embodiment of the present application, the channel strategy configuration is imported into the system operation through the environment variable. The system divides the data block size according to the size of the bandwidth value measured by each.

[0294] Below, Fig.16 Taking as an example, the model training scenario of the attribute processing method of an embodiment of the present application is described, wherein the model training process includes forward propagation and backward propagation.

[0295] In the forward propagation, it includes:

[0296] In the distribution phase, GPU_1 distributes data to other GPUs (GPU_2-GPU_4).

[0297] In the replication phase, the model is distributed to other GPUs;

[0298] In the parallel forward operation phase, multiple threads are started to perform forward calculations;

[0299] In the aggregation phase, the computation output is aggregated to GPU_1.

[0300] exist Fig.16 In the example, the data is divided into four parts: i1, i2, i3, and i4, and the four parts are distributed to GPU_1-GPU_4 respectively. GPU_1-GPU_4 respectively perform forward calculations based on the distributed data to obtain outputs o1, o2, o3, and o4, and respectively aggregate the obtained outputs to GPU_1.

[0301] In the forward propagation, it includes:

[0302] Loss gradient calculation stage, GPU_1 calculates loss gradient;

[0303] Loss gradient distribution stage: distribute the loss gradient to other GPUs.

[0304] In the backward operation phase, each GPU back-propagates the received loss gradients to calculate the gradients of all parameters of the model;

[0305] In the gradient reduction phase, the parameter gradients calculated by all GPUs are aggregated to GPU_1.

[0306] exist Fig.16 , the losses obtained based on o1, o2, o3, and o4 include: loss 1, loss 2, loss 3, and loss 4. GPU_1 obtains loss gradients based on the losses: gradient 1, gradient 2, gradient 3, and gradient 4, and distributes the four loss gradients to GPU_1-GPU_4 respectively. GPU_1-GPU_4 perform backward calculations based on the distributed loss gradients to obtain output parameter gradients: gradient m1, gradient m2, gradient m3, and gradient m4, and aggregate the obtained parameter gradients to GPU_1 respectively.

[0307] In the embodiment of the present application, by combining the positions of GPU and NUMA distribution, combining different physical links, and changing the virtual topology between GPUs, the interconnection between GPUs under different paths is opened up, thereby enhancing the overall processing capability of the system and having scalability. In addition, by building communication channels based on different physical links, users can customize channel operation strategies, balance the bandwidth capabilities of various channels, and allocate data to different communication channels, thereby enhancing the overall throughput capability.

[0308] In the third aspect, in order to implement the above data processing method, an electronic device according to an embodiment of the present application is provided. The structural diagram of the electronic device is described below. Fig.17A and Fig. 17B As shown, the electronic device 1700 includes: a first NUMA node 1701, a second NUMA node 1702 and a processor 1703; wherein, as Fig.17AAs shown, the processor 1703 is located in the first NUMA node 1701 or the second NUMA node 1702, or, as shown in FIG. Fig. 17B As shown, the processor 1703 is independent of the first NUMA node 1701 and the second NUMA node 1702; the first NUMA node 1701 includes a first GPU, the second NUMA node 1702 includes a second GPU, and a first path from the first GPU to the second GPU supports at least two paths, and in different paths, the connection modes of the first NUMA node and the second NUMA node are different;

[0309] Processor 1703, configured to determine that first data is transmitted on a first path, and then obtain transmission performance information of a first data block size of the first data;

[0310] The processor 1703 is further configured to determine a scheduling strategy based on the transmission performance information of the first data block size, where the scheduling strategy is used to characterize a transmission mode of transmitting the first data through the at least two paths;

[0311] The processor 1703 is further configured to schedule, based on the scheduling policy, the transmission of the first data from the first GPU to the second GPU through the at least two paths.

[0312] In some embodiments, the processor 1703 is further configured to:

[0313] Obtaining a configured scheduling policy associated with the first data block size;

[0314] The configured scheduling strategy is adjusted according to the transmission performance information to obtain the scheduling strategy.

[0315] In some embodiments, the processor 1703 is further configured to:

[0316] Acquire a first number of second load information of a first thread, where the first thread is a thread that transmits the first data, and the second load information is load information of the first thread when transmitting a data block of the first data block size;

[0317] Based on the first number of the second load information, the first load information included in the transmission performance information is determined.

[0318] In some embodiments, the at least two pathways include at least two of the following pathways:

[0319] a first path, in which the first GPU is connected to the second GPU via a bounce buffer;

[0320] a second path, in which the first GPU is connected to the second GPU via an NTB;

[0321] A third path, in which the first CPU in the first NUMA node is connected to the second CPU in the second NUMA node through a quick path interconnect (QPI).

[0322] In some embodiments, the processor 1703 is further configured to:

[0323] Determine at least one target path among the at least two paths according to the data allocation ratio of each path among the at least two paths included in the scheduling strategy, and determine an access address corresponding to each target path among the at least one target path according to the data allocation ratio of each target path among the at least one target path and a storage location of the first data in a memory of the first GPU, wherein the target path is a path used for transmission of the first data;

[0324] For each target path in the at least one target path, based on an access address corresponding to the target path, second data is scheduled to be transmitted from the first GPU to the second GPU through the target path, where the second data is data in the first data transmitted using the target path.

[0325] In some embodiments, the processor 1703 is further configured to:

[0326] Determining, according to the size of the first data and the data allocation ratio of each target path in the at least one target path, the data size of the second data used for transmission by each target path in the at least one target path in the first data;

[0327] For each target path in the at least one target path, an access address corresponding to the target path is determined according to a data size of second data used to transmit the target path and a first address of the first data, wherein the first address is a starting address of a storage location of the first data in a memory of the first GPU.

[0328] In some embodiments, the processor 1703 is further configured to:

[0329] Reading the second data from the access address corresponding to the target path, and copying the read second data to a shared memory area, wherein the shared memory area is a mapping of a bounce buffer or an NTB in a CPU memory of a second NUMA node;

[0330] Scheduling the second GPU to read the second data from the shared memory area;

[0331] Schedule the second GPU to copy the second data to the memory of the second GPU.

[0332] In some embodiments, the processor 1703 is further configured to:

[0333] scheduling the bounce buffer to send a notification to the second GPU when the second data in the first shared memory area is available, wherein the notification is used to instruct the second GPU to read the second data;

[0334] The second GPU is scheduled to read the second data from the shared memory area in response to the notification.

[0335] In some embodiments, the processor 1703 is further configured to:

[0336] Before reading the second data from the access address corresponding to the target path, initialize the shared memory in the CPU memory and create a mapping from the bounce buffer or the NTB to the shared memory area in the shared memory;

[0337] After scheduling the second GPU to copy the second data to the memory of the second GPU, releasing the mapping of the bounce buffer or the NTB to the shared memory area.

[0338] In some embodiments,

[0339] The second GPU is used to process the first data based on the running model to obtain a first output result; the first GPU and the second GPU run the same model;

[0340] The first GPU is used to process the third data based on the running model to obtain a second output result;

[0341] The processor 1703 is further configured to schedule the first data output result to be transmitted to the first GPU;

[0342] The first GPU is further used to obtain fourth data based on the first output result and the second output result, and the fourth data is used to determine the loss or to update the parameters of the model.

[0343] It should be noted that the antenna control device provided in the embodiment of the present application includes the various units included, which can be implemented by a combination of a processor in an electronic device and multiple antennas; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU, Central Processing Unit), a microprocessor (MPU, Micro Processor Unit), a digital signal processor (DSP, Digital Signal Processor) or a field programmable gate array (FPGA, Field-Programmable Gate Array), etc.

[0344] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0345] It should be noted that in the embodiment of the present application, if the above-mentioned data processing method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific combination of hardware and software.

[0346] In the third aspect, to implement the above-mentioned data processing method, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the steps in the data processing method provided in the above-mentioned embodiment are implemented.

[0347] In a fourth aspect, an embodiment of the present application provides a storage medium, that is, a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the data processing method provided in the above embodiment are implemented.

[0348] It should be noted here that the description of the above storage medium and device embodiments is similar to the description of the above method embodiments, and has similar beneficial effects as the method embodiments. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the description of the method embodiments of this application for understanding.

[0349] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0350] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0351] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0352] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0353] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0354] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0355] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0356] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, the method being applied to an electronic device, the electronic device comprising: processor; The method comprises: The processor determines to transmit first data on a first path, and then obtains transmission performance information of a first data block size of the first data, wherein the first path is from a first graphics processor GPU to a second GPU, the first GPU is located at a first non-uniform memory access NUMA node, the second GPU is located at a second NUMA node, the first path supports at least two paths, and in different paths, the first NUMA node and the second NUMA node are connected in different ways; The processor determines a scheduling strategy based on the transmission performance information of the first data block size, wherein the scheduling strategy is used to characterize a transmission mode of transmitting the first data through the at least two paths; The processor schedules the first data to be transmitted from the first GPU to the second GPU through the at least two paths based on the scheduling policy.

2. The method according to claim 1, wherein determining a scheduling strategy based on the transmission performance information of the first data block size comprises: Obtaining a configured scheduling policy associated with the first data block size; The configured scheduling strategy is adjusted according to the transmission performance information to obtain the scheduling strategy.

3. The method according to claim 1, wherein the transmission performance information comprises first load information, and the method further comprises: Acquire a first number of second load information of a first thread, where the first thread is a thread that transmits the first data, and the second load information is load information of the first thread when transmitting a data block of the first data block size; The first load information is determined based on the first number of the second load information.

4. The method according to claim 1, wherein the at least two paths include at least two of the following paths: a first path, in which the first GPU is connected to the second GPU via a bounce buffer; a second path, in which the first GPU is connected to the second GPU via a non-transparent bridge NTB; A third path, in which the first CPU in the first NUMA node is connected to the second CPU in the second NUMA node via a quick path interconnect (QPI).

5. The method according to any one of claims 1 to 4, wherein the scheduling strategy comprises a data allocation ratio of each of the at least two paths; Scheduling the transmission of the first data from the first GPU to the second GPU through the at least two paths based on the scheduling policy includes: Determine at least one target path among the at least two paths according to a data allocation ratio of each path among the at least two paths, and determine an access address corresponding to each target path among the at least one target path according to the data allocation ratio of each target path among the at least one target path and a storage location of the first data in a memory of the first GPU, wherein the target path is a path used for transmission of the first data; For each target path in the at least one target path, based on an access address corresponding to the target path, second data is scheduled to be transmitted from the first GPU to the second GPU through the target path, where the second data is data in the first data transmitted using the target path.

6. The method according to claim 5, wherein determining the access address corresponding to each target path in the at least one target path according to the data allocation ratio of each target path in the at least one target path and the storage location of the first data in the memory of the first GPU comprises: Determining, according to the size of the first data and the data allocation ratio of each target path in the at least one target path, the data size of the second data used for transmission by each target path in the at least one target path in the first data; For each target path in the at least one target path, an access address corresponding to the target path is determined according to a data size of second data used to transmit the target path and a first address of the first data, wherein the first address is a starting address of a storage location of the first data in a memory of the first GPU.

7. The method according to claim 5, wherein the electronic device comprises the second NUMA node, and the step of scheduling the second data to be transmitted from the first GPU to the second GPU through the target path based on the access address corresponding to the target path comprises: Reading the second data from the access address corresponding to the target path, and copying the read second data to a shared memory area, wherein the shared memory area is a mapping of a bounce buffer or an NTB in a CPU memory of a second NUMA node; Scheduling the second GPU to read the second data from the shared memory area; Schedule the second GPU to copy the second data to the memory of the second GPU.

8. The method according to claim 7, wherein scheduling the second GPU to read the second data from the shared memory area comprises: scheduling the bounce buffer to send a notification to the second GPU when the second data in the first shared memory area is available, wherein the notification is used to instruct the second GPU to read the second data; The second GPU is scheduled to read the second data from the shared memory area in response to the notification.

9. The method according to any one of claims 1 to 4, wherein the electronic device comprises the first NUMA node and the second NUMA node, and the first GPU and the second GPU run the same model, and the method further comprises: The second GPU processes the first data based on the running model to obtain a first output result; The first GPU processes the third data based on the running model to obtain a second output result; The processor schedules the first output result to be transmitted to the first GPU; The first GPU obtains fourth data based on the first output result and the second output result, and the fourth data is used to determine the loss or to update the parameters of the model.

10. An electronic device, comprising a first non-uniform memory access NUMA node, a second NUMA node and a processor; wherein: The processor is located in the first NUMA node or the second NUMA node, or the processor is independent of the first NUMA node and the second NUMA node; the first NUMA node includes a first graphics processor GPU, the second NUMA node includes a second GPU, a first path from the first GPU to the second GPU supports at least two paths, and in different paths, the first NUMA node and the second NUMA node are connected in different ways; The processor is configured to determine that first data is transmitted on a first path, and then obtain transmission performance information of a first data block size of the first data; The processor is further configured to determine a scheduling strategy based on the transmission performance information of the first data block size, wherein the scheduling strategy is used to characterize a transmission mode of transmitting the first data through the at least two paths; The processor is further configured to schedule, based on the scheduling policy, the transmission of the first data from the first GPU to the second GPU through the at least two paths.

Images