Model weight parameter synchronization method and device

Abstract

The present disclosure provides a model weight parameter synchronization method and device, relates to the field of artificial intelligence, and particularly relates to the field of reinforcement learning. The specific implementation scheme is as follows: determining a transmission path between a sending end and a receiving end according to local topology information of the sending end and local topology information of the receiving end; in response to detecting that the sending end sends a model weight parameter, storing the model weight parameter in a buffer; and in response to a predetermined condition being met, synchronizing the model weight parameter in the buffer to the receiving end through the transmission path.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and more particularly to the field of reinforcement learning, specifically to a method and apparatus for synchronizing model weight parameters. Background Technology

[0002] In the iterative process of Reinforcement Learning from Human Feedback (RLHF), after the training framework completes the weight iteration update for each training step, it needs to synchronize the latest model parameters to the inference framework in real time. The inference framework then performs inference sampling based on the latest model parameters, and the generated sampling data is fed back into the training process to support the next round of reinforcement learning training iteration. Summary of the Invention

[0003] This disclosure provides a method, apparatus, device, storage medium, and computer program product for synchronizing model weight parameters.

[0004] According to a first aspect of this disclosure, a method for synchronizing model weight parameters is provided, comprising: determining a transmission path between a transmitter and a receiver based on local topology information of a transmitter and local topology information of a receiver; storing the model weight parameters in a buffer in response to detecting that the transmitter is sending model weight parameters; and synchronizing the model weight parameters in the buffer to the receiver through the transmission path in response to satisfying a predetermined condition.

[0005] According to a second aspect of this disclosure, a model weight parameter synchronization device is provided, comprising: a determining unit configured to determine a transmission path between a transmitter and a receiver based on local topology information of a transmitter and local topology information of a receiver; an aggregation unit configured to store model weight parameters in a buffer in response to detecting that the transmitter is sending model weight parameters; and a synchronization unit configured to synchronize the model weight parameters in the buffer to the receiver through the transmission path in response to satisfying a predetermined condition.

[0006] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in any one of the first aspects.

[0007] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in any one of the first aspects.

[0008] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in any one of the first aspects.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0010] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is an exemplary system architecture diagram to which one embodiment of this disclosure can be applied; Figure 2 This is a flowchart of an embodiment of the model weight parameter synchronization method according to this disclosure; Figure 3 This is a schematic diagram of an application scenario of the model weight parameter synchronization method according to this disclosure; Figure 4 This is a flowchart of yet another embodiment of the model weight parameter synchronization method according to this disclosure; Figure 5 This is a schematic diagram of a structure of an embodiment of the model weight parameter synchronization device according to the present disclosure; Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0011] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0012] Figure 1 An exemplary system architecture is shown that can be applied to embodiments of the model weight parameter synchronization method or model weight parameter synchronization apparatus of this disclosure.

[0013] like Figure 1 As shown, the system architecture may include a transmitter, a receiver, and a communication library module.

[0014] The sending end deploys a training framework to provide updated model weight parameters after each training step, submitting these parameters by calling the sending interface of the communication library module. The receiving end deploys an inference framework, calling the receiving interface of the communication library module to retrieve the weight parameters transmitted by the sending end, allowing the inference framework to load the model and perform inference sampling.

[0015] The sender and receiver can be located on the same GPU or on different GPUs. Different transmission paths are selected depending on whether the transmission crosses GPUs.

[0016] This application encapsulates the model weight synchronization function into a communication library module independent of the training and inference frameworks. This communication library module exposes only send and receive interfaces, transmitting transmission path configuration information through a unified metadata object that contains no framework-specific information. Model format conversion (such as reconstructing from parallel fragments in the training framework to a complete tensor) and loading the received model are both performed outside the communication library module, allowing it to be reused between any training and inference frameworks. Based on this decoupled architecture, the core technical steps described below are implemented.

[0017] The control logic (channel establishment, topology discovery, metadata transfer) of the communication library module is executed on the CPU side, while the data plane (bucket buffer allocation, tensor aggregation, load transfer) is executed on the GPU to which the process is bound.

[0018] It should be noted that the model weight parameter synchronization method provided in the embodiments of this disclosure is generally executed by the communication library module, and correspondingly, the model weight parameter synchronization device is generally set in the communication library module.

[0019] Continue to refer to Figure 2 The diagram illustrates a flow 200 of an embodiment of a model weight parameter synchronization method according to the present disclosure. This model weight parameter synchronization method includes the following steps: Step 201: Determine the transmission path between the sender and receiver based on the local topology information of the sender and the receiver.

[0020] In this embodiment, the execution body of the model weight parameter synchronization method (e.g.) Figure 1The communication library module shown obtains the local topology information of the sending and receiving ends, which may include: process global rank (rank), physical GPU ID (gpu_id), node IP address (ip), and role identifier (role) (sender or receiver). It determines whether the receiver is a same-GPU receiver as the sender based on whether they are on the same GPU. For same-GPU receivers, IPC (Inter-Process Communication) can be selected as the transmission path. For cross-GPU receivers, aggregated communication can be selected as the transmission path, such as NCCL (NVIDIA Collective Communications Library).

[0021] Step 202: In response to detecting that the sender is sending model weight parameters, store the model weight parameters in a buffer.

[0022] In this embodiment, when the sending end needs to send model weight parameters, it calls the sending interface, for example, send((name, tensor)), where name is the name of the model weight parameter and tensor is the content of the model weight parameter, i.e., a tensor. When the sending end calls the sending interface, it does not immediately send the tensor, but instead merges multiple smaller tensors into a larger "bucket" for unified communication, thereby reducing the number of communication initiations and improving network bandwidth utilization.

[0023] In distributed training, models typically contain hundreds or thousands of parameter tensors. If each tensor initiates a separate communication (such as AllReduce or Send / Recv), it incurs significant communication startup overhead, and small packet transmissions cannot effectively utilize high-bandwidth networks. By aggregating parameters into buckets according to certain rules, with each bucket serving as a communication unit, the number of communication operations can be significantly reduced.

[0024] The buffer is located in the GPU and can be a one-dimensional flat uint8 buffer (default capacity 1 GiB, configurable).

[0025] Step 203: In response to the fulfillment of predetermined conditions, the model weight parameters in the buffer are synchronized to the receiving end through the transmission path.

[0026] In this embodiment, it is continuously monitored whether a predetermined condition is met, and the predetermined condition includes at least one of the following: Condition 1: The amount of model weight parameters stored in the buffer has reached a preset threshold (e.g., 80% of the buffer capacity). Condition 2: The time interval since the last synchronization reaches a preset time threshold (e.g., 10 milliseconds). Condition 3: All model weight parameters in the current training iteration have been written to the buffer; Condition 4: Receive the parameter synchronization request signal sent by the receiving end.

[0027] In response to the fulfillment of any of the aforementioned predetermined conditions, the communication library module packages the model weight parameters in the buffer and sends them to the receiving end via the transmission path determined in step 201. Upon receiving the data, the receiving end writes it into the corresponding weight parameter location of its local model and returns an acknowledgment message to the sending end. After receiving the acknowledgment message, the sending end releases the buffer space occupied by the synchronized data.

[0028] Existing solutions either send data one frame at a time (high overhead) or serialize it to CPU memory or disk (low bandwidth).

[0029] The method provided in the above embodiments of this disclosure dynamically determines the optimal transmission path based on the local topology information of both parties, which can make full use of available network bandwidth. By temporarily storing model weight parameters in the GPU buffer and synchronizing them in batches according to predetermined conditions, without going through CPU memory or file system, the PCIe bandwidth bottleneck is avoided, and the number of network transmissions is reduced, communication overhead is lowered, thereby improving the overall efficiency of parameter synchronization in distributed training.

[0030] In some optional implementations of this embodiment, determining the transmission path between the sender and receiver based on the sender's local topology information and the receiver's local topology information includes: obtaining the sender's local topology information and the receiver's local topology information, wherein the local topology information includes: process global ranking, GPU number, node IP address, and sender / receiver identifier; determining a global topology view based on the sender's local topology information and the receiver's local topology information; for each receiver, determining whether the receiver is located on the same GPU as the sender based on the global topology view; in response to being located on the same GPU, determining that the transmission path between the sender and the receiver is inter-process communication; in response to being located on different GPUs, determining that the transmission path between the sender and the receiver is set communication.

[0031] Each participating process in the system (including the sending end training process and the receiving end inference process) constructs local topology information RankInfo, which includes: process global ranking rank, physical GPU number gpu_id, node IP address ip, and role identifier role (SENDER or RECEIVER).

[0032] All participating processes perform an all_gather collection communication operation once, enabling each process to obtain a global topology view.

[0033] The partition_receivers partitioning algorithm is executed as follows: a sender lookup table is built using the (ip, gpu_id) tuple as the key; for each receiver, if its (ip, gpu_id) matches a sender in the lookup table, the receiver is assigned to that sender's IPC allocation table; otherwise, the receiver is added to the NCCL receiver list. If the same (ip, gpu_id) corresponds to multiple senders, an exception is thrown immediately to expose configuration errors as early as possible.

[0034] This embodiment constructs a global topology view containing process ranking, GPU ID, node IP, and transceiver identifiers, enabling topology-aware adaptive transmission path selection. For transceivers within the same GPU, inter-process communication (IPC) is used for direct memory-level data exchange, avoiding unnecessary network protocol stack overhead and serialization latency. For transceivers across GPUs, aggregated communication (such as NCCL) is used to achieve efficient data transmission using hardware-optimized communication primitives. This differentiated path decision-making mechanism can automatically match the optimal communication method based on the actual deployment topology, reducing intra-device communication latency while fully leveraging the bandwidth advantages of cross-device aggregated communication, thereby improving the overall efficiency and throughput of model weight parameter synchronization in the distributed system.

[0035] In some optional implementations of this embodiment, storing the model weight parameters in the buffer includes: flattening the tensor of the model weight parameters into a continuous byte sequence; in response to the length of the continuous byte sequence being less than or equal to the remaining space in the buffer, copying the continuous byte sequence to the remaining space via device-to-device, and generating a tensor description table, wherein the tensor description table includes: the name of the tensor, the original shape of the tensor, the offset position of the buffer, and the data type of the tensor.

[0036] Maintain a GPU-side one-dimensional flat uint8 buffer (default capacity 1 GiB, configurable).

[0037] Each time send((name, tensor)) is called, the tensor is flattened into a continuous sequence of bytes, copied to the current offset position of the buffer via Device-to-Device, and a TensorSpec metadata entry is recorded (i.e., the tensor description table, which can contain the tensor's name, the tensor's original shape, the offset position of the buffer, and the tensor's data type).

[0038] When the accumulated number of bytes reaches or exceeds the bucket capacity threshold, a flush operation is automatically triggered. For a single tensor exceeding the threshold, the bucket size is dynamically expanded.

[0039] When the sender calls send(None, flush=True), it performs a final flush and sets the stream end flag is_end=True to indicate the end of the stream.

[0040] This embodiment flattens the model weight parameter tensor into a continuous byte sequence and writes it directly to the remaining space of the buffer using a device-to-device (D2D) copy method. This avoids the extra overhead of data transfer through CPU memory and significantly reduces the latency of data transfer between video memory. At the same time, by generating a tensor description table containing tensor name, original shape, buffer offset position and data type, multiple tensors can be compactly stored in the same continuous buffer and can be accurately located and restored. This not only improves the memory utilization of the buffer, but also ensures that the receiving end can correctly restore the continuous byte sequence to the original tensor based on the description information. Thus, an efficient, secure and traceable model weight parameter buffer storage mechanism is achieved.

[0041] In some optional implementations of this embodiment, the method further includes: in response to the length of the continuous byte sequence being greater than the remaining space, synchronizing the model weight parameters in the buffer to the receiving end through the transmission path, releasing the buffer, and deleting the tensor description table; copying the continuous byte sequence to the starting position of the buffer through device-to-device copying, and generating a new tensor description table.

[0042] Upon receiving a new tensor of model weight parameters to be stored from the sender, flatten it into a continuous byte sequence and calculate its length, `tensor_bytes`. Compare `tensor_bytes` with the remaining space in the current buffer, `remaining_size`. If tensor_bytes ≤ remaining_size, then write to the remaining space and update the tensor description table in the usual way; If tensor_bytes > remaining_size, the following buffer flush and reset process is triggered: In response to the detection that tensor_bytes > remaining_size, all model weight parameter data from the start position to the write pointer in the current buffer are immediately synchronized to the receiving end through the transmission path determined in the previous embodiment (such as inter-process communication within the same GPU or collection communication across GPUs).

[0043] During synchronization, the continuous data in the buffer is sent together with or associated with the current tensor description table, so that the receiving end can correctly restore the continuous byte sequence into the named model weight parameter tensors according to the offset position, original shape and data type in the tensor description table.

[0044] After confirming that the receiving end has successfully received the data (or the data has been successfully submitted to the communication queue), the sending end performs the following cleanup operations: resets the write pointer of the buffer to the starting position of the buffer (offset 0), logically releasing the entire buffer space so that the buffer can be rewritten; optionally, performs zero-filling on the buffer to clear old data; deletes the current tensor description table, clearing all metadata records of synchronized tensors in it, to avoid confusion between old metadata and newly written data.

[0045] After the buffer is freed, a new sequence of consecutive bytes is copied device-to-device (D2D) from the beginning of the buffer (offset 0). Once the write is complete, the write pointer is updated to the `tensor_bytes` position. Simultaneously, a new tensor description table is generated.

[0046] In one alternative implementation, if the length of the new continuous byte sequence tensor_bytes is greater than the total size of the buffer, the tensor is first split into multiple fragments. The first fragment is written to the refreshed buffer and synchronized. Subsequent fragments repeat the refresh-write process until the entire tensor is fully synchronized.

[0047] In another alternative implementation, the buffer adopts a circular buffer structure. When the remaining space is insufficient, a portion of the new data can be written to the remaining space, triggering synchronization, and then the remaining portion can be written to the starting position, thereby reducing the waste of effective data caused by whole-block refresh.

[0048] In this embodiment, when the remaining space in the buffer is insufficient to accommodate new model weight parameters, a "synchronization-release-rewrite" refresh mechanism is automatically triggered: First, the parameters accumulated in the buffer are synchronized to the receiving end in batches to avoid data overflow and loss; then, the buffer is reset and the old tensor description table is deleted, achieving efficient circular reuse of memory space without frequent allocation and release of large blocks of contiguous memory; finally, the new parameters are written from the beginning of the buffer and a completely new tensor description table is generated, ensuring that the metadata of the new batch of data is accurate and traceable. This mechanism not only ensures the continuity and security of parameter writing, but also reduces the number of communication startups through batch synchronization, while enabling the limited buffer space to support the continuous buffering and synchronization of any number and size of model weight parameters, significantly improving the throughput and memory utilization efficiency of parameter transmission in distributed training.

[0049] In some optional implementations of this embodiment, the method further includes: dynamically expanding the buffer in response to the length of the continuous byte sequence being greater than the length of the buffer; writing the continuous byte sequence to the expanded buffer via device-to-device copy; and generating a tensor description table, wherein the tensor description table includes: the name of the tensor, the original shape of the tensor, the offset position of the buffer, and the data type of the tensor.

[0050] In actual training, the model may contain large parameter tensors (such as embedding layer parameters or fully connected layer weights in a large language model), whose flattened byte length may exceed the total capacity of the buffer. A new total buffer length is determined based on a pre-defined expansion strategy. The expansion strategy can be any of the following: On-demand expansion: Ensures that new tensors can be stored while reserving some remaining space; Multiplicative expansion: For example, if it is doubled, and it is still less than the number of bytes in the parameter tensor, then continue to multiply until the condition is met; Alignment extension: Aligns the new size upwards to memory page boundaries or GPU memory allocation granularity (such as 2MB alignment) to improve memory access efficiency.

[0051] After the buffer is expanded, the sending end copies the new consecutive byte sequence via device-to-device (D2D). Once the write is complete, the tensor descriptor table is generated or updated.

[0052] This embodiment addresses scenarios where the flattened length of a single model weight parameter tensor exceeds the total buffer capacity. Through a dynamic buffer expansion mechanism, the sending end can adaptively adjust the buffer size, overcoming the initial buffer capacity limitation and ensuring that parameter tensors of any size can be completely stored in a contiguous buffer without fragmentation. This mechanism eliminates the need for pre-allocation of buffer space for the largest parameter size, saving normal GPU memory usage while flexibly handling occasional large parameters in the model, thus improving the robustness and resource utilization of the distributed training parameter synchronization system.

[0053] In some optional implementations of this embodiment, the receiving end includes a receiving end located on the same GPU as the sending end; and in response to satisfying a predetermined condition, synchronizing the model weight parameters in the buffer to the receiving end through a transmission path includes: in response to satisfying the predetermined condition, constructing a control frame, wherein the control frame includes a stream end flag, a tensor description table, and the number of payload elements in the buffer; obtaining an inter-process communication handle for the buffer; sending the inter-process communication handle and the control frame to the receiving end on the same GPU in a point-to-point manner, wherein the receiving end on the same GPU directly accesses the buffer through the inter-process communication handle and parses the model weight parameters from the data in the buffer according to the control frame; and releasing the buffer in response to the end of the receiving end on the same GPU accessing the buffer.

[0054] Construct a MetaFrame control frame, which may include: is_end (stream end flag), tensor_specs (list of tensor descriptions), and payload_numel (number of payload elements).

[0055] IPC path (same as GPU): The CUDA IPC interface is called to obtain the IPC handle of the sender's bucket buffer; the handle header, serialized IPC handle, and MetaFrame are sent to the same GPU receiver in a point-to-point manner. The receiver opens the sender's GPU memory through the shared memory interface and reads it directly without data copying.

[0056] The receiving end receives the weight stream in iterator mode using the get_iterator() method.

[0057] The IPC or NCCL receiving path is automatically selected based on the topology partitioning results.

[0058] For each received bucket, iterate through the list of TensorSpecs in the MetaFrame: slice the payload by offset and number of bytes; restore the slice to its original data type and shape through view operations; perform clone() to generate an independent copy, ensuring that the tensors held downstream are not affected by the sender's bucket buffer reclamation.

[0059] The iterator terminates normally when a MetaFrame with is_end=True is received.

[0060] This embodiment addresses the scenario of multi-process parameter synchronization on the same GPU. It achieves cross-process shared mapping of buffer memory through inter-process communication (IPC) handles, enabling the receiving end to directly access the sending end's buffer data without any data copying, thus achieving true zero-copy communication. This significantly reduces latency and memory bandwidth overhead in inter-process parameter synchronization on the same GPU. Control frames are transmitted separately from the actual data, sending only a small IPC handle and metadata (stream end flag, tensor description table, and payload element count) through the control channel. This avoids bottlenecks caused by large amounts of data passing through the control channel. The tensor description table provides the receiving end with a complete basis for accurately locating and reconstructing each named tensor from the continuous byte stream. The buffer is released promptly after the receiving end finishes accessing the data, ensuring the safe reclamation and recycling of memory resources. This mechanism fully utilizes the hardware characteristics of shared GPU memory, significantly improving the efficiency of model weight parameter synchronization in distributed training while ensuring data integrity and traceability.

[0061] In some optional implementations of this embodiment, the receiving end includes a cross-GPU receiving end located on a different GPU than the sending end; and in response to satisfying a predetermined condition, synchronizing the model weight parameters in the buffer to the receiving end via a transmission path includes: in response to satisfying the predetermined condition, constructing a control frame, wherein the control frame includes a stream end flag, a tensor description table, and the number of payload elements in the buffer; broadcasting the control frame to the cross-GPU receiving end via set communication; broadcasting the data in the buffer to the cross-GPU receiving end via set communication, and releasing the buffer, wherein the cross-GPU receiving end parses the model weight parameters from the received data in the buffer according to the control frame.

[0062] Construct a MetaFrame control frame, which includes: is_end (stream end flag), tensor_specs (list of tensor descriptions), and payload_numel (number of payload elements).

[0063] NCCL path (cross-GPU): First, broadcast the MetaFrame control frame (so that the receiver knows the load size and pre-allocates the receive buffer), then broadcast the entire bucket load tensor.

[0064] The receiving end receives the weight stream in iterator mode using the get_iterator() method.

[0065] The IPC or NCCL receiving path is automatically selected based on the topology partitioning results.

[0066] For each received bucket, iterate through the list of TensorSpecs in the MetaFrame: slice the payload by offset and number of bytes; restore the slices to their original data type and shape; perform clone() to generate independent copies, ensuring that the tensors held downstream are not affected by the sender's bucket buffer reclamation.

[0067] The iterator terminates normally when a MetaFrame with is_end=True is received.

[0068] This embodiment achieves two-stage cross-GPU broadcasting of control frames and buffer data through set communication. First, a control frame containing a stream end flag, tensor description table, and payload element count is broadcast, allowing each receiver to know the data layout in advance and pre-allocate a receive buffer. Then, the actual parameter data is broadcast, and the weight parameters of each named model are accurately restored based on the tensor description table. The set communication primitive fully utilizes the high-speed interconnection and hardware optimization capabilities between GPUs to support efficient one-to-many transmission. After the broadcast is completed, the buffer is immediately released to achieve memory circular reuse. Thus, in cross-GPU scenarios, efficient, reliable, and traceable multi-receiver synchronization of model weight parameters is achieved with controllable communication overhead.

[0069] In some optional implementations of this embodiment, the predetermined conditions include: the sending end no longer sending model weight parameters or the number of bytes of model weight parameters in the buffer is greater than a predetermined threshold; and when the sending end no longer sends model weight parameters, the value of the stream end flag is 1, so that the receiving end stops receiving model weight parameters; and when the number of bytes of model weight parameters in the buffer is greater than the predetermined threshold, the value of the stream end flag is 0.

[0070] In this embodiment, by setting the stream end flag to 0, it is indicated that this is only an intermediate batch transmission. The receiving end maintains its receiving state and continues to wait for subsequent data frames, avoiding misjudging the termination of the data stream due to a single synchronization. When the sending end confirms that it will no longer send parameters, synchronization is triggered, and the stream end flag is set to 1, indicating that this is the last frame of the current data stream. The receiving end stops receiving and completes the end processing of this parameter synchronization, avoiding resource consumption caused by continuous invalid listening. The two triggering conditions reuse the same buffer synchronization process, and flow control semantics can be transmitted through only a single flag bit. While ensuring the efficiency of batch transmission, it enables the receiving end to accurately perceive the data stream boundary and accurately control the synchronization lifecycle, ensuring the integrity and reliability of multi-batch parameter synchronization.

[0071] Lifecycle management can include: transport paths are managed using a process-level singleton pattern, supporting reuse and forced rebuilding. The sending end releases the bucket buffer and reclaims shared memory after each flush.

[0072] This application is applied to large-scale RLHF (Reinforcement Learning Human Feedback) training systems, with specific application areas and processes as follows: Application Scenario 1: Weight synchronization in colocate mode (1) In this mode, the training process and the inference process are deployed on the same GPU pool, and there is a sender (training process) and a receiver (inference process) on the same physical GPU at the same time.

[0073] (2) After the training step is completed, the framework first unloads the gradients and optimizer states on the training side to the CPU, but retains the model parameters on the GPU.

[0074] (3) Load the weight container of the inference engine onto the GPU.

[0075] (4) Perform weight synchronization. At this time, the sending end and the receiving end are located on the same physical GPU. The system automatically selects the IPC zero-copy path. After the sending end aggregates the parameters into the bucket buffer, it only transmits the IPC memory handle (KB level) through the CPU side channel. The receiving end directly maps the GPU memory of the sending end with the handle and copies it to its own weight space. There is no cross-device data transmission throughout the process.

[0076] (5) After synchronization is complete, the training parameters are unloaded to the CPU, the KV cache of the inference framework is loaded to the GPU, and the next round of inference sampling begins.

[0077] Application Scenario 2: Weight Synchronization in Fully Asynchronous Mode

[0078] (1) In this mode, training and inference each occupy independent GPUs, and there is no sender / receiver pair with the same physical GPU.

[0079] (2) After the training step is completed, weight synchronization is performed directly. The system automatically selects the NCCL broadcast path and transmits bucket load data through high-speed interconnect (such as NVLink).

[0080] Application Scenario 3: Fault Tolerance and Recovery Scenario

[0081] (1) When the inference framework fails and is recovered by the scheduler, the physical deployment of the new process may change (e.g., it may be scheduled to a different GPU).

[0082] (2) By forcibly rebuilding the channel, topology discovery is re-executed to adapt to the new physical deployment topology.

[0083] (3) The reconstruction process is transparent to the training side, which only perceives the change in the total number of global processes participating in the distributed task.

[0084] Application Area Expansion

[0085] (1) This application can be widely applied to all scenarios that require hot updates of model weights between training and inference, including but not limited to: RLHF training, online distillation, continuous pre-training, multi-model collaborative training, etc.

[0086] (2) This application is not limited to a specific model architecture and is applicable to various large language models such as Transformer and MoE (hybrid expert). As long as the training side can derive the weights in the form of (name, tensor) pairs and the inference side can receive them in the same interface.

[0087] See also Figure 3 , Figure 3 This is a schematic diagram illustrating an application scenario of the model weight parameter synchronization method according to this embodiment. Figure 3 In this application scenario, the weight synchronization process of this system is a three-stage pipeline, from the generation of tensors at the training end to the writing of tensors into the model at the inference end, and the entire process follows the design concept of "buffering and aggregation → topology-aware routing → differentiated transmission → on-demand unpacking". The following is a complete process description: ① Send Path Step 1: Tensor Production The core model on the training side holds the slice parameters and generates tensors one by one through the generator.

[0088] Step 2: Tensor loading

[0089] Each produced tensor calls the API, such as send(name, tensor), into the send channel. Then, the following operations are performed: the tensor is flattened into a continuous sequence of bytes; and written to a bucket buffer on the GPU side via device-to-device (D2D) copy.

[0090] Step 3: Trigger refresh

[0091] Construct a control frame to send the data in the current bucket via the IPC or NCCL path.

[0092] ② Transport Channel

[0093] The transport layer comprises three communication groups (Group 1, Group 2, and Group 3) and a topology partitioning algorithm. Step 4: Divide the receiving end into two types of transmission paths according to the topology partitioning algorithm: Receiver with the same SIM card (same IP address and GPU ID as sender) → uses the IPC path. For receivers with different IP addresses or GPU IDs across different cards, the NCCL path is used. ③ Recv Path Step 5: Receive buffer data The receiving end obtains the iterator by calling the `get_iterator()` interface, receives the buffer data, and unpacks the data. Depending on the transmission path, there are two receiving methods: same-card IPC reception (via shared memory) and cross-card NCCL reception (via broadcast communication).

[0094] Step 6: Restore the weight parameters and generate independent copies

[0095] Regardless of the transmission path, after unpacking the data, the element width of each tensor is restored according to the data type, and the multidimensional dimensions are restored according to the original shape to obtain the restored weight parameters. An independent copy of the restored weight parameters is generated (especially in IPC scenarios, to break away from shared buffer dependencies).

[0096] The restored weight parameters are written to the model weights in the inference framework via the entry point. The model loading and synchronization are completed internally within the inference framework. The cache is cleared after each round.

[0097] Further reference Figure 4 This illustrates a flow 400 of another embodiment of the model weight parameter synchronization method. Flow 400 of the model weight parameter synchronization method includes the following steps: Step 401: Create a first communication group, a second communication group, and a third communication group that are independent of the training framework at the sending end and the inference framework at the receiving end.

[0098] In this embodiment, the underlying API of the distributed communication library is used to create communication groups that are independent of the existing default process groups of the training and inference frameworks. Each communication group uses an independent TCPStore instance and an independent port to rendezvous.

[0099] The first communication group, also known as the main Gloo group (all participants, used for coordination and IPC handle passing, port offset 0), is a communication backend in PyTorch (distinct from NCCL). It runs on the CPU and is suitable for control plane / initialization tasks. In actual frameworks (such as Megatron, DeepSpeed, and PyTorch DDP), a separate Gloo process group is typically created specifically for running `all_gather` to exchange topology information during the initialization phase. After the topology is built, the NCCL group is used for actual high-performance data communication.

[0100] The second communication group, also known as the NCCL data group (sender rank and all NCCL receivers, used to transmit tensor payload, port offset 1).

[0101] The third communication group, also known as the NCCL metadata group (same participant, Gloo backend, used for broadcast control frames, port offset 2).

[0102] Step 402: Determine the transmission path between the sender and receiver using the first communication group based on the local topology information of the sender and the receiver.

[0103] In this embodiment, the first communication group is used to allow all processes to exchange information about "which machine and which card I am on" to create a global topology map. Subsequent communications can then be optimized based on this map for hierarchical optimization within / between machines and within / between cards.

[0104] Choose either the IPC or NCCL transmission path based on the global topology map. See step 201 for details.

[0105] Step 403: In response to detecting that the sender is sending model weight parameters, store the model weight parameters in the buffer.

[0106] Step 403 is basically the same as step 202, so it will not be described again.

[0107] Step 404: In response to the fulfillment of predetermined conditions, the model weight parameters in the buffer are synchronized to the receiving end via the transmission path through the second communication group and the third communication group.

[0108] In this embodiment, a MetaFrame control frame is constructed, which includes: is_end stream end flag, tensor_specs list of tensor descriptions, and payload_numel number of payload elements.

[0109] IPC path (same as GPU): The CUDA IPC interface is called to obtain the IPC handle of the sender's bucket buffer; the handle header, serialized IPC handle, and MetaFrame are sent to the receiver on the same GPU via the main Gloo group in a point-to-point manner. The receiver opens the sender's GPU memory through the shared memory interface and reads it directly without data copying.

[0110] NCCL path (cross-GPU): First, broadcast the MetaFrame control frame through the NCCL metadata group (so that the receiver knows the load size and pre-allocates the receive buffer), and then broadcast the entire bucket load tensor through the NCCL data group.

[0111] When shutting down, the NCCL data group, NCCL metadata group, and main Gloo group are destroyed in sequence, and all internal states are cleared.

[0112] In this embodiment, by creating a communication group independent of the existing default process group, complete decoupling of any training framework and inference framework is achieved; existing solutions usually require the sender and receiver to belong to the same process group or the same framework.

[0113] This application abstracts weight synchronization into an independent communication library module, independent of any training framework's parallel primitives or inference framework's internal state management interface. Both the sender and receiver access the channel through a unified channel metadata configuration object, which contains only general parameters such as network address and number of participants, without any framework-specific information. In contrast, existing solutions deeply couple weight transmission logic with specific training / inference frameworks, requiring rewriting the synchronization logic when switching between frameworks. This application's decoupled design allows the weight synchronization channel to be deployed and reused independently as a general component.

[0114] This application contributes to the product in the following ways: Performance improvements: Bucket aggregation reduces the number of communication rounds from hundreds to thousands, comparable to the number of parameter tensors, to single digits (for example, a 7B parameter model with approximately 14GB of weights requires only about 14 flushes for a 1GB bucket), significantly improving effective bandwidth utilization. When deployed on the same GPU, zero-copy IPC is used, resulting in near-zero transmission latency.

[0115] Deployment flexibility: Topology-aware adaptive path selection enables the same code to support both shared GPU deployment and separate deployment, eliminating the need for users to write different weight synchronization logic for different deployment modes.

[0116] Framework Decoupling: This invention abstracts weight synchronization into an independent communication library unrelated to the training / inference framework. The sending end only needs to provide iterators of (name, tensor) pairs, and the receiving end only needs to consume iterators of the same format. The training side can use Megatron, DeepSpeed, or any other framework, while the inference side can use SGLang, vLLM, or any engine that supports per-tensor loading. Both sides only need to follow the ChannelMeta channel configuration protocol to communicate with each other without modifying any internal framework code.

[0117] Fault tolerance: Supports channel reconstruction mechanism, which can automatically adapt to new topology configuration after the inference engine fails and recovers, without manual intervention.

[0118] Resource efficiency: Streaming buckets avoids holding complete model copies simultaneously, reducing peak GPU memory usage; bucket buffers are released and shared memory is reclaimed immediately after sending.

[0119] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a model weight parameter synchronization device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0120] like Figure 5 As shown, the model weight parameter synchronization device 500 of this embodiment includes: a determining unit 501, an aggregation unit 502, and a synchronization unit 503. The determining unit 501 is configured to determine the transmission path between the sending end and the receiving end based on the local topology information of the sending end and the local topology information of the receiving end; the aggregation unit 502 is configured to store the model weight parameters in a buffer in response to detecting that the sending end is transmitting model weight parameters; and the synchronization unit 503 is configured to synchronize the model weight parameters in the buffer to the receiving end through the transmission path in response to meeting predetermined conditions.

[0121] In this embodiment, the specific processing of the determination unit 501, aggregation unit 502, and synchronization unit 503 of the model weight parameter synchronization device 500 can be referred to Figure 2 Steps 201, 202, and 203 in the corresponding embodiment.

[0122] In some optional implementations of this embodiment, the determining unit 501 is further configured to: obtain local topology information of the sending end and local topology information of the receiving end, wherein the local topology information includes: global process ranking, GPU number, node IP address, and transceiver identifier; determine a global topology view based on the local topology information of the sending end and the local topology information of the receiving end; for each receiving end, determine whether the receiving end is located on the same GPU as the sending end based on the global topology view; in response to being located on the same GPU, determine that the transmission path between the sending end and the receiving end is inter-process communication; in response to being located on different GPUs, determine that the transmission path between the sending end and the receiving end is set communication.

[0123] In some optional implementations of this embodiment, the aggregation unit 502 is further configured to: flatten the tensor of the model weight parameters into a continuous byte sequence; in response to the length of the continuous byte sequence being less than or equal to the remaining space in the buffer, copy the continuous byte sequence to the remaining space via device-to-device, and generate a tensor description table, wherein the tensor description table includes: the name of the tensor, the original shape of the tensor, the offset position of the buffer, and the data type of the tensor.

[0124] In some optional implementations of this embodiment, the aggregation unit 502 is further configured to: in response to the length of the continuous byte sequence being greater than the remaining space, synchronize the model weight parameters in the buffer to the receiving end through the transmission path, release the buffer, delete the tensor description table; copy the continuous byte sequence to the starting position of the buffer through device-to-device copying, and generate a new tensor description table.

[0125] In some optional implementations of this embodiment, the aggregation unit 502 is further configured to: dynamically expand the buffer in response to the length of the continuous byte sequence being greater than the length of the buffer; write the continuous byte sequence to the expanded buffer via device-to-device copy; and generate a tensor description table, wherein the tensor description table includes: the name of the tensor, the original shape of the tensor, the offset position of the buffer, and the data type of the tensor.

[0126] In some optional implementations of this embodiment, the synchronization unit 503 is further configured to: construct a control frame in response to a predetermined condition being met, wherein the control frame includes a stream end flag, a tensor description table, and the number of load elements in the buffer; obtain the inter-process communication handle of the buffer; send the inter-process communication handle and the control frame to the card receiver in a point-to-point manner, wherein the card receiver directly accesses the buffer through the inter-process communication handle and parses the model weight parameters from the data in the buffer according to the control frame; and release the buffer in response to the card receiver ending its access to the buffer.

[0127] In some optional implementations of this embodiment, the synchronization unit 503 is further configured to: construct a control frame in response to a predetermined condition being met, wherein the control frame includes a stream end flag, a tensor description table, and the number of load elements in the buffer; broadcast the control frame to the cross-card receiver via set communication; broadcast the data in the buffer to the cross-card receiver via set communication, and release the buffer, wherein the cross-card receiver parses the model weight parameters from the data in the received buffer according to the control frame.

[0128] In some optional implementations of this embodiment, the predetermined conditions include: the sending end no longer sending model weight parameters or the number of bytes of model weight parameters in the buffer is greater than a predetermined threshold; and when the sending end no longer sends model weight parameters, the value of the stream end flag is 1, so that the receiving end stops receiving model weight parameters; and when the number of bytes of model weight parameters in the buffer is greater than the predetermined threshold, the value of the stream end flag is 0.

[0129] In some optional implementations of this embodiment, the apparatus 500 further includes a creation unit (not shown in the figures), configured to: create a first communication group, a second communication group, and a third communication group independent of the training framework of the sending end and the inference framework of the receiving end. The first communication group is used to determine a global topology view based on the local topology information of the sending end and the local topology information of all receiving ends, and to send inter-process communication handles and control frames to the same-card receiving end in a point-to-point manner. The second communication group is used to broadcast data in the buffer to the cross-card receiving end through aggregated communication. The third communication group is used to broadcast control frames to the cross-card receiving end through aggregated communication.

[0130] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0131] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0132] An electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described in process 200 or 400.

[0133] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in process 200 or 400.

[0134] A computer program product includes a computer program that, when executed by a processor, implements the method described in process 200 or 400.

[0135] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0136] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the electronic device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0137] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0138] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as the model weight parameter synchronization method. For example, in some embodiments, the model weight parameter synchronization method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the model weight parameter synchronization method described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the model weight parameter synchronization method by any other suitable means (e.g., by means of firmware).

[0139] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or electronic device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or electronic devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0143] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0144] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0145] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0146] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for synchronizing model weight parameters, comprising: The transmission path between the sending end and the receiving end is determined based on the local topology information of the sending end and the local topology information of the receiving end. In response to detecting that the sending end is sending model weight parameters, the model weight parameters are stored in a buffer; In response to the fulfillment of predetermined conditions, the model weight parameters in the buffer are synchronized to the receiving end through the transmission path.

2. The method according to claim 1, wherein, Determining the transmission path between the sender and the receiver based on the local topology information of the sender and the receiver includes: Obtain the local topology information of the sending end and the local topology information of the receiving end, wherein the local topology information includes: global process ranking, GPU number, node IP address, and transceiver identifier; A global topology view is determined based on the local topology information of the sending end and the local topology information of the receiving end; For each receiving end, it is determined whether the receiving end and the sending end are located on the same GPU based on the global topology view; in response to being located on the same GPU, the transmission path between the sending end and the receiving end is determined to be inter-process communication; in response to being located on different GPUs, the transmission path between the sending end and the receiving end is determined to be set communication.

3. The method according to claim 1, wherein, The step of storing the model weight parameters into the buffer includes: Flatten the tensor of the model weight parameters into a continuous byte sequence; In response to the length of the continuous byte sequence being less than or equal to the remaining space in the buffer, the continuous byte sequence is copied to the remaining space via device-to-device and a tensor description table is generated, wherein the tensor description table includes: the name of the tensor, the original shape of the tensor, the offset position in the buffer, and the data type of the tensor.

4. The method according to claim 3, further comprising: In response to the length of the continuous byte sequence being greater than the remaining space, the model weight parameters in the buffer are synchronized to the receiving end through the transmission path, the buffer is released, and the tensor description table is deleted; The continuous byte sequence is copied from device to device to the beginning of the buffer, and a new tensor description table is generated.

5. The method according to claim 3, further comprising: The buffer is dynamically expanded in response to the length of the consecutive byte sequence being greater than the length of the buffer. The continuous byte sequence is copied to the extended buffer via device-to-device copying, and a tensor description table is generated, wherein the tensor description table includes: the name of the tensor, the original shape of the tensor, the offset position of the buffer, and the data type of the tensor.

6. The method according to claim 3, wherein, The receiving end includes a card-based receiving end located on the same GPU as the sending end; and The step of synchronizing the model weight parameters in the buffer to the receiving end through the transmission path in response to the fulfillment of a predetermined condition includes: In response to the fulfillment of predetermined conditions, a control frame is constructed, wherein the control frame includes a stream end flag, the tensor description table, and the number of load elements in the buffer; Obtain the inter-process communication handle of the buffer; The inter-process communication handle and the control frame are sent to the same card receiving end in a point-to-point manner. The same card receiving end directly accesses the buffer through the inter-process communication handle and parses the model weight parameters from the data in the buffer according to the control frame. Upon completion of access to the buffer by the same card receiving end, the buffer is released.

7. The method according to claim 3, wherein, The receiving end includes a cross-card receiving end located on a different GPU than the sending end; as well as The step of synchronizing the model weight parameters in the buffer to the receiving end through the transmission path in response to the fulfillment of a predetermined condition includes: In response to the fulfillment of predetermined conditions, a control frame is constructed, wherein the control frame includes a stream end flag, the tensor description table, and the number of load elements in the buffer; The control frame is broadcast to the cross-card receiver via aggregated communication; The data in the buffer is broadcast to the cross-card receiver via aggregated communication, and the buffer is released. The cross-card receiver parses the model weight parameters from the data in the received buffer according to the control frame.

8. The method according to claim 6 or 7, wherein, The predetermined conditions include: the sending end no longer sending model weight parameters or the number of bytes of model weight parameters in the buffer is greater than a predetermined threshold; and when the sending end no longer sends model weight parameters, the value of the stream end flag is 1, so that the receiving end stops receiving model weight parameters; and when the number of bytes of model weight parameters in the buffer is greater than the predetermined threshold, the value of the stream end flag is 0.

9. The method according to claim 1, wherein, The receiving end includes a same-card receiving end located on the same GPU as the sending end and / or a cross-card receiving end located on a different GPU than the sending end; and The method further includes: A first communication group, a second communication group, and a third communication group are created, independent of the training framework of the sending end and the inference framework of the receiving end. The first communication group is used to determine a global topology view based on the local topology information of the sending end and the local topology information of all receiving ends, and to send inter-process communication handles and control frames to the same-card receiving end in a point-to-point manner. The second communication group is used to broadcast the data in the buffer to the cross-card receiving end through aggregate communication. The third communication group is used to broadcast control frames to the cross-card receiving end through aggregate communication.

10. A model weight parameter synchronization device, comprising: The determining unit is configured to determine the transmission path between the sending end and the receiving end based on the local topology information of the sending end and the local topology information of the receiving end. An aggregation unit is configured to store the model weight parameters into a buffer in response to detecting that the sender is transmitting model weight parameters. The synchronization unit is configured to synchronize the model weight parameters in the buffer to the receiving end through the transmission path in response to a predetermined condition.

11. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-9.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-9.

Images