Data transmission method and system suitable for distributed training and switch

Abstract

The invention discloses a data transmission method and system suitable for distributed training and a switch, the data transmission method directly realizes multicast operation through the switch, and comprises the following steps: setting a multicast enable bit and a multicast extension field in a TLP data packet, if the multicast enable bit in the TLP data packet is a target field, sending the multicast extension field to the switch, and if the multicast enable bit in the TLP data packet is the target field, sending the multicast extension field to the switch. If yes, analyzing the multicast extension field by using an analysis module in the switch to determine a position parameter of the multicast object target equipment, and determining a port corresponding to the position parameter as a target port; and executing the following multicast operations by utilizing a preset multicast copying module: determining a target port as a main port, determining other target ports as slave ports, writing the TLP data packet into a temporary storage area of the main port, and based on a first-in first-out principle, writing the TLP data packet into the temporary storage area of the main port. And synchronously copying the data in the temporary storage area of the master port to the temporary storage area of each slave port by adopting a DMA controller in the switch. According to the invention, high-efficiency multicast operation can be realized, and link occupation and system overhead are reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer and communication technologies, and in particular to a data transmission method, system and switch suitable for distributed training. Background Art

[0002] In AI training systems, PCIe (PCI Express) is commonly used as the interconnect bus between the CPU and GPU / TPU, both from the host to the accelerator and between accelerators. PCIe is essentially a point-to-point layered protocol: each link can only maintain reliable transmission between a pair of endpoints, and there is no link-level multicast or broadcast semantics.

[0003] Therefore, when the training framework needs to execute "one-to-many" or "many-to-many" collective communication semantics - such as the common broadcast (used to synchronize the latest parameters on the master node or a master parameter server to all computing nodes), all-reduce (used by multiple nodes to collaborate on reduction operations such as parameter addition or averaging), and all-gather (used to centrally collect the results calculated by multiple child nodes, such as gradients) in distributed data parallel training, a logical multicast operation can only be split into multiple independent point-to-point DMA (Direct Memory Access) transactions.

[0004] like Figure 1 As shown in the figure, a typical topology with a switch uplink serving a single CPU and N GPUs / TPUs serving as downlinks is used as an example. Parallel and distributed computing often require synchronizing gradients across all accelerators, necessitating 2×(N−1) unidirectional transfers using an all-reduce operation. In existing PCIe switching networks, sequential or concurrent point-to-point data transmission inevitably occupies both the uplink and downlink links. Therefore, the "one-to-many" or "many-to-many" data transmission methods employed by existing PCIe architectures suffer from at least the following drawbacks: (1) Link bandwidth amplification: Each gradient synchronization requires the actual transmission of N−1 copies of data on the PCIe link. The overall link occupancy increases linearly in O(N). When N reaches 8, 16, or even higher, the link bandwidth is quickly exhausted by redundant traffic. (2) DMA engine overhead amplification: The DMA engine of each GPU / TPU must generate independent descriptors for each pair of endpoints and independently maintain context such as interrupts and error retransmissions. The number of descriptors is proportional to N², and the DMA queue depth and host CPU interrupt load increase sharply. (3) Delay superposition: Since the PCIe switch chip uses a scheduling mechanism based on port arbitration, multi-port concurrent DMA is prone to problems such as queue congestion and head-of-line blocking, further extending the communication delay and reducing the training iteration throughput.

[0005] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of the present application, nor does it necessarily provide technical guidance. In the absence of clear evidence that the above content has been disclosed before the filing date of the present application, the above background technology should not be used to evaluate the novelty and creativity of the present application. Summary of the Invention

[0006] The purpose of the present invention is to provide a data transmission method, system and switch suitable for distributed training, which can maintain transparency to the existing training framework, achieve efficient multicast operations, and significantly reduce link occupancy and communication delays.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows: A data transmission method suitable for distributed training, which directly implements multicast operations through switches, includes the following steps: Setting a multicast enable bit and a multicast extension field in a TLP data packet. For a TLP data packet that needs to be transmitted to multiple target devices through a switch, the multicast enable bit is the target field, and the multicast extension field includes a multicast target bit, which is configured to record location parameters of the target devices. checking, using an ingress parser of the switch, whether the multicast enable bit in the TLP packet is a destination field; if so, parsing the multicast extension field to determine the location parameter using a parsing module pre-set in the switch; and determining, by the parsing module, a port of the switch corresponding to the location parameter as the destination port; The following multicast operations are performed using a multicast replication module pre-installed in the switch: one target port is determined as the master port and the other target ports are determined as slave ports, the TLP data packet is written into the temporary storage area of ​​the master port, and based on the first-in-first-out principle, the DMA controller in the switch is used to synchronously copy the data in the temporary storage area of ​​the master port to the temporary storage areas of each of the slave ports.

[0008] Further, according to any one of the above technical solutions or a combination of multiple technical solutions, the location parameter includes a BDF parameter for determining the location of the target device, and the BDF parameter includes a bus number, a device number, and a function number; Each port of the switch is configured with a unique port ID, and a device port mapping table is pre-configured in the switch, the device port mapping table including a one-to-one correspondence between BDF parameters and port IDs; The parsing module determines the target port by querying the port ID corresponding to the BDF parameter in the device port mapping table.

[0009] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the switch includes multiple ports, each port is configured with a unique port ID, and the device connected to the port is configured with a device ID, and the device ID has a corresponding relationship with the port ID; The location parameters include a target device ID and / or a target port ID; The parsing module determines the target port according to the target device ID and / or the target port ID.

[0010] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the multicast enable bit is set in a reserved bit of the TLP data packet header.

[0011] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, the multicast extension field further includes a start flag bit, the start flag bit is configured as the starting position of the multicast extension field, and the multicast extension field parsing module determines the starting position of the multicast extension field by identifying the start flag bit; and / or, The multicast extension field further includes a multicast number bit, which is configured to record the total number of target devices corresponding to the multicast operation, and the multicast extension field parsing module determines the total number of target devices corresponding to the multicast operation by parsing the multicast number bit; and / or, The multicast extension field further includes a version bit, wherein the version bit is configured to record a version number of the multicast extension field, and the multicast extension field determines the version number of the multicast extension field by parsing the version bit; and / or, The multicast extension field further includes an F bit set after each field bit. If the F bit is 1, the multicast extension field does not end; if the F bit is 0, the multicast extension field ends. and / or, The multicast extension field further includes a check bit, which is configured to record a check mode of the TLP data packet. The multicast extension field parsing module determines the check mode of the TLP data packet by parsing the check bit.

[0012] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, if the ingress parser of the switch determines that the multicast enable bit in the TLP data packet is not a destination field, and the switch determines that the TLP data packet is a broadcast data packet, the following operations are performed: A group matcher pre-set in the switch is used to determine a broadcast port and a corresponding broadcast address according to a preset broadcast group table and transmit the determined broadcast port and the corresponding broadcast address to a preset broadcast module in the switch, wherein the broadcast group table includes a memory address configured for each of the ports, the broadcast port is a port other than the request port, and the broadcast address is one or more of the memory addresses; The broadcast module transmits the TLP data packet to each of the broadcast ports through the broadcast address according to the sum.

[0013] Furthermore, any one of the above technical solutions or a combination of multiple technical solutions further includes the following steps: Pre-setting a shared memory in the switch, the shared memory including a plurality of memory addresses, configuring a corresponding memory address for each port, and the port being configured to communicate with its corresponding memory address; The memory address corresponding to the broadcast port is determined to be the broadcast address, and the broadcast module copies the broadcast data into multiple copies corresponding to the number of broadcast ports and transmits the copies to each of the broadcast addresses.

[0014] Furthermore, any one of the above technical solutions or a combination of multiple technical solutions may further include the following steps: Pre-setting a shared memory in the switch, the shared memory including a plurality of memory addresses, configuring a corresponding memory address for each port, and the port being configured to communicate with any of the memory addresses; Determine that the memory address corresponding to the request port is the broadcast address.

[0015] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, if the ingress parser of the switch determines that the multicast enable bit in the TLP data packet is not a destination field, and the switch determines that the TLP data packet is a non-broadcast data packet, the following operations are performed: Determine a target device corresponding to the TLP data packet, determine a target port of a switch corresponding to the target device, and transmit the TLP data packet to the target port.

[0016] Furthermore, any one of the above technical solutions or a combination of multiple technical solutions may further include the following steps: Pre-setting a shared memory in the switch, the shared memory including a plurality of memory addresses, configuring a corresponding memory address for each port, and the port being configured to communicate with the memory address; Determine the memory address corresponding to the target port as the target memory address, and transmit the TLP data packet to the target memory address.

[0017] According to another aspect of the present invention, a communication system is provided, comprising a switch and a plurality of devices electrically connected to the switch, wherein the plurality of devices transmit data to other devices through the switch based on a data transmission method suitable for distributed training as described in any one of the above technical solutions or a combination of multiple technical solutions.

[0018] According to another aspect of the present invention, a switch suitable for distributed training is provided, comprising a parsing module and a multicast replication module. In response to receiving a multicast enable signal, the parsing module parses a multicast extension field in a current TLP packet to determine location parameters corresponding to each target device, and the parsing module determines a port of the switch corresponding to the location parameter as a target port. The multicast replication module determines one target port as a master port and determines the other target ports as slave ports; The switch writes the TLP data packet into the temporary storage area of ​​the master port, and based on the first-in-first-out principle, the multicast replication module controls the DMA controller in the switch to synchronously copy the data in the temporary storage area of ​​the master port to the temporary storage areas of each of the slave ports.

[0019] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, it also includes a broadcast replication engine electrically connected to each of the ports of the switch, and the broadcast replication engine includes a group matcher and a broadcast module electrically connected; In response to determining that a communication request initiated by a request port is a broadcast request, the broadcast replication engine sends the communication request to the group matcher and receives a TLP data packet transmitted by the request port target; In response to receiving the broadcast request, the group matcher determines a broadcast port and a corresponding broadcast address according to a preset broadcast group table and transmits the broadcast port and the corresponding broadcast address to the broadcast module, wherein the broadcast group table includes a memory address configured for each port, the broadcast port is a port other than the request port, and the broadcast address is one or more of the memory addresses; In response to receiving the broadcast port and the broadcast address, the broadcast module controls each of the broadcast ports to receive the TLP data packet through the broadcast address.

[0020] The beneficial effects brought about by the technical solution provided by the present invention are as follows: a. The data transmission method for distributed training proposed in the present invention determines one of the target ports as a master port and the other target ports as slave ports. While transmitting TLP packets to the master port's buffer, the method utilizes a DMA-based bypass channel within the switch to synchronously transfer data from the master port's buffer to the buffers of each slave port based on the first-in-first-out principle. This method eliminates the need to create multiple TLP packets and implement multicast operations through a system bus or primary data path. Furthermore, the multicast process is completely isolated from the primary path, preventing interference with primary path scheduling and eliminating the need for additional refresh / invalidation operations. This method effectively improves multicast efficiency and reduces data transmission latency. b. The proposed definition of the multicast enable bit and multicast extension field enables mapping multiple destination addresses through a single descriptor, enabling consistent multicast / broadcast protocols across devices. This approach also complies with existing communication system architectures and training framework definitions, does not disrupt existing framework data flow scheduling rules, and can be directly applied to PCIe topologies. c. The present invention implements data transmission, broadcast operation and P2P communication from the requesting device that initiates the TLP data packet to the master port based on shared memory, which can not only improve the efficiency of multicast operation, but also improve the efficiency of P2P communication and broadcast operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 Schematic diagram of the one-to-many data transmission principle in the existing PCIe architecture; Figure 2 A flowchart of a data transmission method applicable to distributed training provided by an exemplary embodiment of the present invention; Figure 3 A block diagram of a switch suitable for distributed training according to an exemplary embodiment of the present invention is provided; Figure 4 A schematic diagram of a multicast operation principle based on DMA and first-in-first-out principle provided for an exemplary embodiment of the present invention; Figure 5 A schematic diagram of a multicast operation process provided as a flow chart of an exemplary embodiment of the present invention; Figure 6 A schematic diagram of the architecture of a communication system provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] In addition to the technical issues discussed in the background, the lack of "multicast" semantic support on existing GPU / TPU hardware is a key obstacle. When generating DMA descriptors, mainstream training frameworks (such as PyTorch and TensorFlow XLA) can only specify the BDF (Bus-Device-Function) of a single target device according to the PCIe specification. Even if the driver layer wants to transparently expand a single logical multicast into multiple point-to-point transmissions, it cannot maintain semantic correctness while avoiding the following issues.

[0026] Address space isolation: The BAR (Base Address Register) windows of each GPU / TPU are independent of each other, and the driver cannot map multiple destination addresses through a single descriptor.

[0027] Cache consistency: The GPU / TPU internal L2 cache and HBM controller lack a cross-device consistency broadcast protocol. If the driver forcibly splits the descriptors, additional flush / invalidate operations are required, further increasing latency.

[0028] Software transparency: The training framework expects to see the abstraction of "one DMA completes multicast". If the driver exposes multiple descriptors upward, it will destroy the framework's stream scheduling assumptions and cause deadlock or data contention.

[0029] In summary, the existing PCIe architecture lacks efficient one-to-many or many-to-many multicast communication semantics at both the hardware protocol layer and the GPU / TPU device end. This leads to serious bandwidth amplification, DMA engine waste, and software stack complexity in distributed AI training scenarios with multiple GPUs on a single machine. While the industry has attempted to partially alleviate these issues with alternative solutions such as NVSwitch, CXL.mem, and RoCEv2, these solutions require either dedicated chips or additional network protocol conversion, making them difficult to reuse in pure PCIe topologies.

[0030] To solve the above technical problems, this application proposes a multicast transmission mechanism that can be directly applied to PCIe switches and existing PCIe architectures, which can maintain transparency to the existing training framework, achieve efficient multicast operations, reduce the communication delay of multicast operations, and significantly reduce link occupancy and DMA descriptor overhead.

[0031] In one embodiment of the present invention, a data transmission method suitable for distributed training is provided. Figure 2 、 Figure 3 and Figure 4 As shown, the data transmission method directly implements multicast operation through a switch, including the following steps: A multicast enable bit and a multicast extended field (MEF) are set in a TLP packet. For a TLP packet that needs to be transmitted to multiple target devices through a switch, the multicast enable bit is the target field, and the multicast extended field includes a multicast target bit, which is configured to record the location parameters of the target devices. checking, using an ingress parser of the switch, whether the multicast enable bit in the TLP packet is a destination field; if so, parsing the multicast extension field to determine the location parameter using a parsing module pre-set in the switch; and determining, by the parsing module, a port of the switch corresponding to the location parameter as the destination port; The following multicast operations are performed using a multicast replication module pre-installed in the switch: one target port is determined as the master port and the other target ports are determined as slave ports, the TLP data packet is written into the temporary storage area of ​​the master port, and based on the first-in-first-out principle, the DMA controller in the switch is used to synchronously copy the data in the temporary storage area of ​​the master port to the temporary storage areas of each of the slave ports.

[0032] In one embodiment of the present invention, in a communication system in which multiple devices are electrically connected via a switch for data transmission, the switch is pre-installed with the parsing module (full name: Multicast Extension Field Parser, MEF Parser), the multicast replication module, and the device port mapping table (BDF - Port Mapping Table). The communication system is initialized, including initializing hardware resources such as DMA channels and CRC.

[0033] One of the devices (such as a GPU) initiates a regular memory write (DMA Write) or command transmission operation, and generates a TLP data packet based on the multicast enable bit and multicast extension field proposed in this application to send to the switch.

[0034] The multicast enable bit is preferably set in a reserved bit in the TLP data packet header. For example, Attributes[7] (reserved bit) in the TLP data packet header is used as the multicast enable bit (MC_En). If MC_En=0, it indicates that the TLP data packet is a non-multicast data packet and does not trigger the multicast parsing / judgment process; if MC_En=1, it indicates that the TLP data packet is a multicast data packet and the data area of ​​the TLP data packet is accompanied by a multicast extension field (MEF), which triggers the multicast parsing / judgment process.

[0035] As shown in Table 1, the multicast extension field includes not only the position parameter, but also field bits such as a start flag bit, a multicast number bit, a version bit, an F bit, a check bit, and a reserved bit.

[0036] The multicast destination includes a BDF parameter used to determine the location of the target device. As shown in the bottom two columns of Table 1, the BDF parameters include a bus number, a device number, and a function number. Each port of the switch is configured with a unique port ID. A device port mapping table (BDF-port mapping table) is preconfigured in the switch. This table contains a one-to-one correspondence between BDF parameters and port IDs. The parsing module queries the device port mapping table for the port ID corresponding to the BDF parameter, which serves as the target port ID. The parsing module then determines that the port corresponding to the target port ID is the target port.

[0037] Wherein, the version bit is configured to record the version number of the multicast extension field, and the multicast extension field determines the version number of the multicast extension field by parsing the version bit. In actual application, the multicast extension field can be updated according to usage requirements and technical iterations. Coding and decoding rules are determined for each version of the multicast extension field. The parsing module parses the multicast extension field based on the coding and decoding rules corresponding to the version number, and the parsing module can also judge the security of the TLP data packet based on the version number. For example, if the parsing module determines that the version number of the multicast extension field does not meet the requirements, it is determined that the TLP data packet has been tampered with, and an error message and prompt information are issued.

[0038]

[0039] The start flag is configured as the start position of the multicast extension field, and the multicast extension field parsing module determines the start position of the multicast extension field by identifying the start flag. The multicast extension field determines the start position of its parsed object according to the start flag.

[0040] The multicast extension field also includes a multicast number bit, which is configured to record the total number of target devices corresponding to the multicast operation. The multicast extension field parsing module determines the total number of target devices corresponding to the multicast operation by parsing the multicast number bit, and can determine the reliability of the multicast data by verifying the consistency between the total number of target devices and the number of bus numbers / device numbers.

[0041] The F bit is set after each field bit, and the parsing module determines whether the multicast extension field ends according to the F bit. For example, if the F bit is 1, the multicast extension field does not end; if the F bit is 0, the multicast extension field ends.

[0042] The check bit is configured to record the check method of the TLP data packet, and the multicast extension field parsing module determines the check method of the TLP data packet by parsing the check bit. For example, in a specific embodiment, the check bit has a width of 2 bits. If the check bit is 00, it indicates that the TLP data packet does not need to be checked; if the check bit is 01, it indicates that the TLP data packet needs to be checked using the CRC16 algorithm; if the check bit is 01, it indicates that the TLP data packet needs to be checked using the 10=CRC32 algorithm.

[0043] The above-mentioned definition method of the multicast enable bit and multicast extension field can map multiple destination addresses through a single descriptor to achieve consistent multicast / broadcast protocols across devices. It does not conflict with the existing communication system architecture and training framework definition, and will not destroy the data flow scheduling rules of the existing framework. It can be directly applied to the PCIe topology architecture.

[0044] The TLP data packet enters the PCIe / CXL switch and is received by the switch's ingress parser. If the ingress parser detects that the multicast enable bit is set to the target field (MC_En=1), it sends a multicast enable signal to the parsing module to trigger the multicast determination process.

[0045] The parsing module completes the following tasks: reading and parsing the attached MEF fields, including start flag check, version check, and CRC verification (if enabled); extracting the target BDF list (Bus, Device, Function) and determining the target number Target Cnt of the multicast operation; querying the internal device port mapping table (BDF-port mapping table) to map each target BDF to the physical output port (port ID) of the switch; summarizing to form a target list (Multicast Tag, McTag), which includes metadata such as the target port list of multiple target ports, the number of target BDFs, the data start address, and the TLP type; and transmitting the target list to the path selector (Path Selector) of the switch and the multicast replication module.

[0046] The path selector determines whether the number of target BDFs is less than the total number of ports of the switch, and if so, cooperates with the multicast replication module to enter the FIFO-to-FIFO mode.

[0047] like Figure 3 and Figure 4 As shown, for example, if device 0 connected to port 0 initiates a TLP packet and transmits it to the switch through port 0, and the multicast targets for the TLP packet are devices 1, 2, 3, and 4, then port 1 corresponding to device 1, port 2 corresponding to device 2, port 3 corresponding to device 3, and port 4 corresponding to device 4 are the target ports. The multicast replication module determines one of ports 1 to 4 as the master port and the others as slave ports.

[0048] like Figure 5As shown, a master port is selected to execute FIFO-to-FIFO, writing the TLP data packet into the master port's FIFO buffer (a temporary storage area based on the first-in, first-out principle). Data is then read from the master port's FIFO buffer and transferred to the FIFO buffers of the remaining slave ports using DMA. Assuming port 1 is the master port, the switch's inherent path selector will transmit the TLP data packet to port 1's temporary storage area (buffer) using port 0 as the send port and port 1 as the receive port. Simultaneously, the multicast replication module controls the switch's DMA controller to use port 1's temporary storage area as the source address and the temporary storage areas of ports 2 through 4 as the destination addresses. Following the first-in, first-out principle, whenever port 1's temporary storage area is full, the switch's DMA controller will transfer the data from port 1's temporary storage area to the temporary storage areas of ports 2 through 4.

[0049] In the above, based on the multicast replication module and DMA controller, one of the target ports is used as the master port to receive TLP packets. As the master port's buffer receives the TLP packets, the DMA controller simultaneously transfers the master port's buffer to the buffers of each slave port. Specifically, whenever the master port's buffer is full, the switch's DMA controller transfers the data from the master port's buffer to the buffers of each slave port. This timely data transmission of small amounts of data based on DMA and the first-in, first-out principle is extremely fast and can effectively reduce communication delays in multicast operations.

[0050] The one-to-many multicast operation proposed by the present invention does not require the creation of multiple TLP packets. It operates by moving data directly from one FIFO read port to another FIFO write port, bypassing the system bus or primary data path. The path through which the master port receives TLP packets is completely isolated from the DMA-based single-shot multicast process. The FIFO-to-FIFO and DMA-based multicast operation is equivalent to establishing a "bypass multicast channel" within the switch. During a "single write" to the master port, "multiple asynchronous copies" are performed simultaneously. The main computing pipeline only perceives the "single write," and subsequent copies are completely transparent to it, without interfering with primary path scheduling. No additional flush / invalidate operations are required, effectively improving multicast efficiency and reducing data transmission latency.

[0051] In one embodiment of the present invention, a shared memory is pre-set in the switch, and the data transmission process from the requesting device to the requesting port, from the requesting port to the main port, and from the main port to the main target device is implemented based on the shared memory. The requesting device is the device that initiates the TLP data packet, and the requesting port is the port in the switch connected to the requesting device.

[0052] Specifically, a shared memory is pre-set in the switch, the shared memory including a plurality of memory addresses, a corresponding memory address is configured for each port, and the port is configured to communicate with the memory address, specifically, to communicate with the physical memory address of the device based on DMA. The memory address corresponding to the master port is determined as the target memory address, and the TLP data packet is transferred from the requesting device to the target memory address based on DMA, and the data in the target memory address is transferred to the temporary storage area of ​​the master port based on DMA.

[0053] For example, port 1 is connected to device 1 and is configured with DMA controller 1 and DMA address 1. The DMA controller 1 transfers the DMA address 1 to the physical memory address of the device 1, or the DMA controller 1 transfers the physical memory address of the device 1 to the DMA address 1.

[0054] If the ingress parser of the switch determines that the multicast enable bit in the TLP data packet is not a target field, and the switch determines that the TLP data packet is a one-to-one transmission data packet, the following operations are performed: determining the target device corresponding to the TLP data packet, determining the target port of the switch corresponding to the target device, transferring the TLP data packet directly from the physical memory address of the requesting device to the target memory address based on DMA, and then transferring the data in the target memory address directly to the physical memory address of the target device connected to the target port based on DMA.

[0055] The data transmission method applicable to distributed training described in the above embodiment is also applicable to broadcast operations. During the broadcast operation, the target port is all ports except the request port.

[0056] In another embodiment of the present invention, a data transmission method suitable for distributed training is provided. The difference from the above embodiment is that in this embodiment, the broadcast operation is not implemented by the multicast operation described in the above embodiment, but rather by an independent broadcast operation different from the multicast operation implementation process. In this embodiment, if the ingress parser of the switch determines that the multicast enable bit in the TLP packet is not the target field, and the switch determines that the TLP packet is a broadcast packet, the following operations are performed: using a pre-set group matcher in the switch to determine a broadcast port and a corresponding broadcast address according to a preset broadcast group table and transmit them to a pre-set broadcast module in the switch, the broadcast group table including memory addresses configured for each of the ports, the broadcast port being a port other than the request port, and the broadcast address being one or more of the memory addresses; the broadcast module transmits the TLP packet to each of the broadcast ports according to the broadcast address.

[0057] There are two ways to implement the broadcast operation based on the shared memory. One way is: the port is configured to communicate with its corresponding memory address, and the port does not communicate with the memory addresses corresponding to other ports; the memory address corresponding to the broadcast port is determined to be the broadcast address, and the broadcast module copies the broadcast data into multiple copies corresponding to the number of broadcast ports and transmits them to each of the broadcast addresses. This data transmission method has high reliability.

[0058] Another way is: the port is configured to communicate with any of the memory addresses; the memory address corresponding to the request port is determined to be the broadcast address. This data transmission method is more efficient and can further reduce the delay of data transmission.

[0059] Another embodiment of the present invention differs from the above embodiment in that: in the multicast extension field, the target port is determined directly based on the target device ID and / or the target port ID, rather than based on the BDF parameter. In this embodiment, the switch includes multiple ports, each port is configured with a unique port ID, and the device connected to the port is configured with a device ID, the device ID corresponding to the port ID; the location parameter includes the target device ID and / or the target port ID; and the parsing module determines the target port based on the target device ID and / or the target port ID.

[0060] In one embodiment of the present invention, a communication system is provided, such as Figure 6 As shown, the communication system includes a switch and multiple devices electrically connected to the switch, and the multiple devices transmit data to other devices through the switch based on the data transmission method applicable to distributed training described in any of the above embodiments. Figure 6As shown, both the upstream switches and the downstream switches of the multiple devices can be set as the switches including the parsing module and the multicast replication module as described in the above embodiment, or only one of the upstream switches and the downstream switches of the multiple devices can be set as the switch including the parsing module and the multicast replication module. Preferably, the downstream switches of the multiple devices are set as the switches including the parsing module and the multicast replication module so that the multicast operation does not need to occupy the uplink bandwidth.

[0061] In one embodiment of the present invention, a switch suitable for distributed training is provided, including a parsing module and a multicast replication module. In response to receiving a multicast enable signal, the parsing module parses the multicast extension field in the current TLP data packet to determine the position parameters corresponding to each target device, and the parsing module determines the port corresponding to the position parameter among the ports of the switch as the target port; the multicast replication module determines one target port as the master port and determines the other target ports as slave ports; the switch writes the TLP data packet into the temporary storage area of ​​the master port, and based on the first-in-first-out principle, the multicast replication module controls the DMA controller in the switch to synchronously copy the data in the temporary storage area of ​​the master port to the temporary storage areas of each of the slave ports.

[0062] In this embodiment, the switch further includes a broadcast replication engine electrically connected to each of the ports, and the broadcast replication engine includes an electrically connected group matcher and a broadcast module. In response to determining that a communication request initiated by a request port is a broadcast request, the broadcast replication engine sends the communication request to the group matcher and receives the TLP data packet transmitted by the request port. In response to receiving the broadcast request, the group matcher determines the broadcast port and the corresponding broadcast address according to a preset broadcast group table and transmits them to the broadcast module. The broadcast group table includes memory addresses configured for each of the ports, the broadcast port is a port other than the request port, and the broadcast address is one or more of the memory addresses. In response to receiving the broadcast port and broadcast address, the broadcast module controls each of the broadcast ports to receive the TLP data packet through the broadcast address.

[0063] It should be noted that the communication system and the switch embodiment suitable for distributed training provided by the present invention have the same inventive concept as the above-mentioned data transmission method embodiment suitable for distributed training. The entire content of the data transmission method embodiment suitable for distributed training is incorporated into the communication system and the switch embodiment suitable for distributed training by introduction.

[0064] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0065] The above is only a specific implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A data transmission method suitable for distributed training, characterized in that: Implementing multicast operations directly through a switch involves the following steps: Setting a multicast enable bit and a multicast extension field in a TLP data packet. For a TLP data packet that needs to be transmitted to multiple target devices through a switch, the multicast enable bit is the target field, and the multicast extension field includes a multicast target bit, which is configured to record location parameters of the target devices. Determining, using an ingress parser of the switch, whether the multicast enable bit in the TLP packet is a destination field, and if so, parsing the multicast extension field using a parsing module pre-set in the switch to determine the location parameter, wherein the parsing module determines a port of the switch corresponding to the location parameter as the destination port; The following multicast operations are performed using a multicast replication module pre-installed in the switch: one target port is determined as the master port and the other target ports are determined as slave ports, the TLP data packet is written into the temporary storage area of ​​the master port, and based on the first-in-first-out principle, the DMA controller in the switch is used to synchronously copy the data in the temporary storage area of ​​the master port to the temporary storage areas of each of the slave ports.

2. The data transmission method suitable for distributed training according to claim 1, characterized in that: The location parameters include BDF parameters for determining the location of the target device, and the BDF parameters include a bus number, a device number, and a function number; Each port of the switch is configured with a unique port ID, and a device port mapping table is pre-configured in the switch, the device port mapping table including a one-to-one correspondence between BDF parameters and port IDs; The parsing module determines the target port by querying the port ID corresponding to the BDF parameter in the device port mapping table.

3. The data transmission method suitable for distributed training according to claim 1, characterized in that: The switch includes a plurality of ports, each port is configured with a unique port ID, and a device connected to the port is configured with a device ID, and the device ID has a corresponding relationship with the port ID; The location parameters include a target device ID and / or a target port ID; The parsing module determines the target port according to the target device ID and / or the target port ID.

4. The data transmission method suitable for distributed training according to claim 1, characterized in that: The multicast enable bit is set in a reserved bit of the TLP data packet header.

5. The data transmission method suitable for distributed training according to claim 1, characterized in that: The multicast extension field further includes a start flag bit, which is configured as the starting position of the multicast extension field, and the multicast extension field parsing module determines the starting position of the multicast extension field by identifying the start flag bit; and / or, The multicast extension field further includes a multicast number bit, which is configured to record the total number of target devices corresponding to the multicast operation, and the multicast extension field parsing module determines the total number of target devices corresponding to the multicast operation by parsing the multicast number bit; and / or, The multicast extension field further includes a version bit, wherein the version bit is configured to record a version number of the multicast extension field, and the multicast extension field determines the version number of the multicast extension field by parsing the version bit; and / or, The multicast extension field further includes an F bit set after each field bit. If the F bit is 1, the multicast extension field does not end; if the F bit is 0, the multicast extension field ends. and / or, The multicast extension field further includes a check bit, which is configured to record a check mode of the TLP data packet. The multicast extension field parsing module determines the check mode of the TLP data packet by parsing the check bit.

6. The data transmission method suitable for distributed training according to claim 1, characterized in that: If the ingress parser of the switch determines that the multicast enable bit in the TLP packet is not a destination field, and the switch determines that the TLP packet is a broadcast packet, the switch performs the following operations: A group matcher pre-set in the switch is used to determine a broadcast port and a corresponding broadcast address according to a preset broadcast group table and transmit the determined broadcast port and the corresponding broadcast address to a preset broadcast module in the switch, wherein the broadcast group table includes a memory address configured for each of the ports, the broadcast port is a port other than the request port, and the broadcast address is one or more of the memory addresses; The broadcast module transmits the TLP data packet to each of the broadcast ports through the broadcast address.

7. The data transmission method suitable for distributed training according to claim 6, characterized in that: The following steps are also included: Pre-setting a shared memory in the switch, the shared memory including a plurality of memory addresses, configuring a corresponding memory address for each port, and the port being configured to communicate with its corresponding memory address; The memory address corresponding to the broadcast port is determined to be the broadcast address, and the broadcast module copies the broadcast data into multiple copies corresponding to the number of broadcast ports and transmits the copies to each of the broadcast addresses.

8. The data transmission method suitable for distributed training according to claim 6, characterized in that: The following steps are also included: Pre-setting a shared memory in the switch, the shared memory including a plurality of memory addresses, configuring a corresponding memory address for each port, and the port being configured to communicate with any of the memory addresses; Determine that the memory address corresponding to the request port is the broadcast address.

9. The data transmission method suitable for distributed training according to claim 1, characterized in that: If the ingress parser of the switch determines that the multicast enable bit in the TLP packet is not a destination field, and the switch determines that the TLP packet is a non-broadcast packet, the switch performs the following operations: Determine a target device corresponding to the TLP data packet, determine a target port of a switch corresponding to the target device, and transmit the TLP data packet to the target port.

10. The data transmission method suitable for distributed training according to claim 1 or 9, characterized in that: The following steps are also included: Pre-setting a shared memory in the switch, the shared memory including a plurality of memory addresses, configuring a corresponding memory address for each port, and the port being configured to communicate with the memory address; Determine the memory address corresponding to the target port as the target memory address, and transmit the TLP data packet to the target memory address.

11. A communication system, characterized in that: It includes a switch and multiple devices electrically connected to the switch, and the multiple devices transmit data to other devices through the switch based on the data transmission method suitable for distributed training as described in claim 1.

12. A switch suitable for distributed training, characterized in that: The system comprises a parsing module and a multicast replication module. In response to receiving a multicast enable signal, the parsing module parses the multicast extension field in the current TLP data packet to determine the location parameters corresponding to each target device, and the parsing module determines the port corresponding to the location parameter among the ports of the switch as the target port. The multicast replication module determines one target port as a master port and determines the other target ports as slave ports; The switch writes the TLP data packet into the temporary storage area of ​​the master port, and based on the first-in-first-out principle, the multicast replication module controls the DMA controller in the switch to synchronously copy the data in the temporary storage area of ​​the master port to the temporary storage areas of each of the slave ports.

13. The switch suitable for distributed training according to claim 12, characterized in that: It also includes a broadcast replication engine electrically connected to each of the ports of the switch, the broadcast replication engine including an electrically connected group matcher and a broadcast module; In response to determining that a communication request initiated by a request port is a broadcast request, the broadcast replication engine sends the communication request to the group matcher and receives a TLP data packet transmitted by the request port target; In response to receiving the broadcast request, the group matcher determines a broadcast port and a corresponding broadcast address according to a preset broadcast group table and transmits the broadcast port and the corresponding broadcast address to the broadcast module, wherein the broadcast group table includes a memory address configured for each port, the broadcast port is a port other than the request port, and the broadcast address is one or more of the memory addresses; In response to receiving the broadcast port and the broadcast address, the broadcast module controls each of the broadcast ports to receive the TLP data packet through the broadcast address.

Images