A low-communication federated training method and system for automatic driving perception model

Abstract

The application discloses a low-communication federated training method and system for an automatic driving perception model, belongs to the technical field of federated training, and is used for solving the technical problem that the prior art cannot consider both matrix parameter update quality and model differential data upload efficiency in a bandwidth-limited vehicle-road cooperation federated perception training scene. The method comprises the following steps: performing double-branch training on the global model parameter of the current round issued by the server based on local data to generate complete model differential data; the complete model differential data is split into matrix branch differential data and ordinary branch differential data, which are respectively combined with corresponding historical compressed residuals; the two combined branch differential data are respectively subjected to differential compression processing to obtain a matrix branch compressed package and an ordinary branch compressed package, and a complete compressed update package is obtained after combination; and the complete compressed update package is uploaded to the server for model differential data reconstruction, and parameter update of the global model is completed.

Description

Technical Field

[0001] This invention relates to the field of federated training technology, and in particular to a low-communication federated training method and system for autonomous driving perception models. Background Technology

[0002] With the evolution of vehicle-road cooperative autonomous driving and edge intelligence technologies, vehicle terminals, roadside units, and regional edge nodes need to jointly train high-precision autonomous driving perception models based on multi-source perception data such as locally collected video, millimeter-wave radar point clouds, and lidar point clouds to cover core tasks such as road target detection, traffic participant identification, and abnormal event warning. To meet the requirements of data privacy compliance and ensuring that raw data does not leave the domain, federated learning has become the mainstream paradigm for distributed model training in vehicle-road cooperative scenarios: a global perception model is distributed by a central server or regional coordination node, and each edge node completes model training locally, only needing to upload model update differential data to achieve global collaborative optimization, without needing to upload raw perception data or local annotation information throughout the process.

[0003] In the aforementioned vehicle-road cooperative federated training scenario, the uplink communication link is the core bottleneck restricting training efficiency and node scale. Uplinks between vehicles and roadside nodes generally suffer from significant bandwidth fluctuations, limited transmission costs, high cross-regional backhaul latency, and short online training windows. Meanwhile, autonomous driving perception models typically contain a large number of two-dimensional matrix parameters, including convolutional layers, fully connected linear layers, and attention projection layers. The number of model parameters continuously increases with performance improvements. If fully accurate model difference data is directly uploaded in each training round, the amount of uploaded data will increase linearly with the model size, not only driving up transmission costs but also extending the synchronization cycle and limiting the number of nodes that can participate in collaborative training.

[0004] In existing technologies, the two technical directions of local matrix update quality optimization and upload communication volume compression are separated, making it impossible to achieve high-quality local updates and low-overhead upload transmission simultaneously in bandwidth-constrained vehicle-road cooperative scenarios, which is difficult to meet the needs of efficient federated training of autonomous driving perception models. Summary of the Invention

[0005] This invention provides a low-communication federated training method and system for autonomous driving perception models, which addresses the following technical problem: In bandwidth-constrained vehicle-road cooperative federated perception training scenarios, existing technologies cannot simultaneously ensure the quality of matrix parameter updates and the efficiency of model differential data upload.

[0006] The embodiments of the present invention adopt the following technical solutions: On one hand, embodiments of the present invention provide a low-communication federated training method for an autonomous driving perception model. The method includes: performing dual-branch training on the global model parameters of the current round issued by the server based on local data to generate complete model difference data; wherein, the dual-branch training includes: training the matrix parameters through the Muon approximate orthogonalization optimizer and training the ordinary parameters through the backup optimizer. The complete model difference data is split into matrix branch difference data and ordinary branch difference data, and then merged with the corresponding historical compressed residuals. The merged two branch difference data are subjected to differential compression processing to obtain matrix branch compressed package and ordinary branch compressed package, which are then merged to obtain complete compressed update package. The complete compressed update package is uploaded to the server for model differential data reconstruction, completing the global model parameter update and entering the next round of training.

[0007] In one feasible implementation, before performing dual-branch training on the global model parameters issued by the server based on local data to obtain complete model difference data, the method further includes: Receive the global model parameters for this round from the server; wherein, the global model parameters for this round are the global model parameters for the current training round selected by the server after initializing the autonomous driving perception model; Based on the structural properties of the global model parameters in this round, they are divided into matrix parameters and ordinary parameters.

[0008] In one feasible implementation, the global model parameters for this round are divided into matrix parameters and ordinary parameters based on their structural properties, specifically including: The two-dimensional matrix after convolution kernel rearrangement, the linear layer weight matrix, the attention projection matrix, and the feedforward network weight matrix in the current round of global model parameters are divided into matrix parameters. Bias parameters, normalization layer parameters, one-dimensional embedding scaling parameters, and small-size parameters that cannot be orthogonalized are classified as ordinary parameters.

[0009] In one feasible implementation, dual-branch training is performed on the current-round global model parameters issued by the server based on local data to generate complete model difference data, specifically including: Based on local data, several training steps are performed on the matrix parameters and ordinary parameters in the current round of global model parameters to obtain the local model parameters; The difference between the local model parameters and the global model parameters in this round is calculated to obtain the complete model difference data.

[0010] In one feasible implementation, the matrix parameters are trained using a Muon approximate orthogonalization optimizer, specifically including: The gradient matrix of the matrix parameters in the current training step is calculated using the Muon approximate orthogonalization optimizer, and the current momentum matrix is ​​updated based on the gradient matrix and the preset momentum coefficients. Construct an estimated update direction matrix based on the gradient matrix and the updated current momentum matrix; The estimated update direction matrix is ​​approximately orthogonalized to obtain the orthogonalized matrix update direction. The matrix parameters are updated based on the weight decay coefficient, the matrix branch-specific learning rate, and the update direction of the orthogonalized matrix, and then the training proceeds to the next step.

[0011] In one feasible implementation, the complete model difference data is split into matrix branch difference data and ordinary branch difference data, and then merged with the corresponding historical compressed residuals, specifically including: Obtain the local matrix branch parameters corresponding to the matrix parameters and the local ordinary branch parameters corresponding to the ordinary parameters from the local model parameters; Calculate the difference between the local matrix branch parameter and the matrix parameter to obtain the matrix branch difference data; calculate the difference between the local ordinary branch parameter and the ordinary parameter to obtain the ordinary branch difference data; The matrix branch differential data is merged with the historical compressed residuals corresponding to the matrix branches in the local cache, and the ordinary branch differential data is merged with the historical compressed residuals corresponding to the ordinary branches in the local cache.

[0012] In one feasible implementation, the differential data of the two merged branches are subjected to differential compression processing to obtain a matrix branch compressed package and a normal branch compressed package. After merging, a complete compressed update package is obtained, which specifically includes: The merged matrix branch difference data is compressed using at least one of the following methods: low-rank decomposition compression, top-k sparsification compression according to matrix blocks, block sparsification compression, or matrix-aware quantization compression, to obtain the matrix branch compressed package. For the merged ordinary branch differential data, compression is performed using at least one of the following methods: b-bit fixed-point quantization compression, unified top-k sparsity compression, random sparsity compression, or entropy coding compression, to obtain the ordinary branch compressed package; wherein, the matrix branch and the ordinary branch are configured with compression strength parameters independently. The matrix branch compressed package is merged and compressed with the ordinary branch compressed package to obtain the complete compressed update package.

[0013] In one feasible implementation, after performing differential compression processing on the merged two branch differential data to obtain a matrix branch compressed package and a normal branch compressed package, the method further includes: The matrix branch compressed package and the ordinary branch compressed package are decompressed and reconstructed to obtain matrix branch reconstructed differential data and ordinary branch reconstructed differential data; The difference between the matrix branch difference data and the matrix branch reconstructed difference data is calculated and written as a new compressed residual into the cache area corresponding to the matrix branch for residual merging in the next round of training. The difference between the difference data of the ordinary branch and the difference data of the reconstructed ordinary branch is calculated and written into the cache area corresponding to the ordinary branch as a new compressed residual for the next round of training residual merging.

[0014] In one feasible implementation, the complete compressed update package is uploaded to the server for model difference data reconstruction to complete the global model parameter update, specifically including: The complete compressed update package obtained from local training is aggregated to the server, where decompression and reconstruction are performed to obtain the reconstructed model difference data corresponding to each client. The reconstructed model difference data of each client is then aggregated based on a preset aggregation rule to obtain global reconstructed model difference data. The global model parameters are then updated, and the next round of training begins. The preset aggregation rule is either the FedAvg weighted average aggregation rule or the DiLoCo outer aggregation rule.

[0015] On the other hand, embodiments of the present invention also provide a low-communication federated training system for an autonomous driving perception model, the system comprising: The dual-branch training module is used to perform dual-branch training on the global model parameters issued by the server based on local data, generating complete model difference data; wherein, the dual-branch training includes: training the matrix parameters through the Muon approximate orthogonalization optimizer, and training the ordinary parameters through the backup optimizer; The compression and update module is used to split the complete model difference data into matrix branch difference data and ordinary branch difference data, and merge them with the corresponding historical compression residuals respectively; perform differential compression processing on the two branch difference data after merging to obtain matrix branch compressed package and ordinary branch compressed package, and merge them to obtain complete compressed update package; The aggregation update module is used to upload the complete compressed update package to the server for model differential data reconstruction, complete the global model parameter update, and enter the next round of training.

[0016] Compared with existing technologies, the low-communication federated training method for autonomous driving perception models provided in this embodiment of the invention has the following beneficial effects: This invention significantly reduces the amount of data uploaded in a single training round by splitting the model differential into matrix branches and ordinary branches and performing differential compression. This effectively alleviates the uplink bandwidth pressure on vehicle-side and roadside nodes, increases the training synchronization frequency under the same bandwidth conditions, or supports more nodes to participate in collaborative training under the same synchronization frequency, adapting to the bandwidth fluctuations and cost constraints of vehicle-road cooperative scenarios.

[0017] By applying Muon's approximate orthogonalization to update the parameters of the two-dimensional matrix, the update direction structure of the model difference before compression is optimized, thereby improving the information density of the difference. Compared with ordinary compressed federated schemes, higher model validation accuracy and lower validation loss can be achieved under the same upload communication budget, avoiding model performance degradation caused by high compression ratio.

[0018] This invention also provides a dedicated compression strategy for matching the parameter structure features of matrix branches and ordinary branches, making full use of the structural characteristics of matrix differences to improve compression efficiency; at the same time, the two branches independently maintain error feedback caches, accumulating the residuals generated in each round of compression to the next round of difference for compensation, effectively correcting the accumulated deviation caused by multiple rounds of compression, and ensuring the accuracy of the update direction during the training process. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 A flowchart of a low-communication federated training method for an autonomous driving perception model provided in an embodiment of the present invention; Figure 2 A schematic diagram of a client-side local optimization and branch compression process provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of Muon branch differential compression data interaction for an autonomous driving perception model provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a low-communication federated training system for an autonomous driving perception model, provided as an embodiment of the present invention. Detailed Implementation

[0020] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0021] This invention provides a low-communication federated training method for autonomous driving perception models, such as... Figure 1 As shown, the low-communication federated training method for autonomous driving perception models specifically includes steps S101-S104: S101. Based on local data, perform dual-branch training on the global model parameters issued by the server for this round to generate complete model difference data; wherein, the dual-branch training includes: training the matrix parameters through the Muon approximate orthogonalization optimizer and training the ordinary parameters through the backup optimizer.

[0022] Specifically, it receives the global model parameters for this round from the server; where the global model parameters for this round are the global model parameters for the current training round selected by the server after initializing the autonomous driving perception model.

[0023] Furthermore, based on the structural properties of the global model parameters in this round, they are divided into matrix parameters and ordinary parameters. Specifically, the two-dimensional matrix after convolution kernel rearrangement, the linear layer weight matrix, the attention projection matrix, and the feedforward network weight matrix in the global model parameters in this round are classified as matrix parameters; the bias parameters, normalized layer parameters, one-dimensional embedding scaling parameters, and small-sized parameters that cannot be orthogonalized are classified as ordinary parameters.

[0024] As a feasible implementation, during the initialization phase, the server initializes the global model W_0 of the vehicle-road cooperative autonomous driving perception model. This model can be CNN, VGG, ResNet, Transformer, ViT, or a neural network containing two-dimensional matrix weights such as convolution, linear, and attention projection. Then, the server sets the number of communication rounds, the number of local training steps, the client sampling strategy, the aggregation weights, and the FedAvg or DiLoCo outer aggregation method.

[0025] On the client side, parameter partitioning rules are established: the two-dimensional matrix after convolution kernel rearrangement, linear layer weights, attention query / key / value / output projection matrices, and feedforward layer matrices are labeled as Muon matrix parameters. Bias, normalization layers, one-dimensional parameters, small-sized special parameters, and parameters unsuitable for orthogonalization are labeled as ordinary parameters. The client initializes momentum states for matrix parameters and AdamW or SGD optimizer states for ordinary parameters. Additionally, the client initializes an error feedback cache for the object to be compressed; preferably, the matrix branch and the ordinary branch maintain different historical compression residuals.

[0026] The system side sets the branch compression ratio or upload budget, including the matrix branch compression configuration rho_M and the normal branch compression configuration rho_B. For example, matrix branches use low rank, block size, or top-k ratio, while normal branches use quantization bit width b, sparsity ratio, or entropy coding configuration.

[0027] Furthermore, based on local data, several training steps are performed on the matrix parameters and ordinary parameters in the current round of global model parameters to obtain local model parameters. The difference between the local model parameters and the current round of global model parameters is calculated to obtain complete model difference data. The client specifies that the matrix parameters enter the Muon approximate orthogonalization optimizer branch, using momentum and approximate orthogonalization operations to generate matrix update directions; ordinary parameters enter the backup optimizer branch. This design allows orthogonalization calculations to focus on matrix weights that are more likely to benefit, while preserving the stable adaptation capability of the AdamW optimizer or SGD backup optimizer for ordinary parameters.

[0028] As a feasible implementation method, the matrix parameters are trained using the Muon approximate orthogonalization optimizer. Specifically, this includes: calculating the gradient matrix of the matrix parameters in the current training step using the Muon approximate orthogonalization optimizer; updating the current momentum matrix based on the gradient matrix and preset momentum coefficients; constructing an estimated update direction matrix based on the gradient matrix and the updated current momentum matrix; performing approximate orthogonalization processing on the estimated update direction matrix to obtain the orthogonalized matrix update direction; updating the matrix parameters based on the weight decay coefficient, the matrix branch-specific learning rate, and the orthogonalized matrix update direction, and then proceeding to the next training step.

[0029] In one embodiment, for each round of global training, the server selects the clients participating in this round of training and distributes the global model parameters W_r and necessary training configuration parameters for this round. Each client receiving the model parameters performs several steps of parameter training based on local data. For matrix parameters, the client constructs update directions based on gradients and momentum and performs Muon or approximately orthogonalization updates. For ordinary parameters, the client uses backup optimizers such as AdamW, SGD, or MomentumSGD for updates. After the client completes local training, it obtains the local model parameters W_i^{local}, and then calculates the complete model difference Delta_i = W_i^{local} - W_r.

[0030] S102. Split the complete model difference data into matrix branch difference data and ordinary branch difference data, and merge them with the corresponding historical compressed residuals respectively.

[0031] Specifically, the local matrix branch parameters corresponding to the matrix parameters and the local ordinary branch parameters corresponding to the ordinary parameters are obtained from the local model parameters.

[0032] Furthermore, the difference between the local matrix branch parameters and the matrix parameters is calculated to obtain matrix branch difference data; the difference between the local ordinary branch parameters and the ordinary parameters is calculated to obtain ordinary branch difference data.

[0033] In one embodiment, Figure 2 A schematic diagram of a client-side local optimization and branch compression process provided in an embodiment of the present invention is shown below. Figure 2 As shown, the client further generates branch differences according to the parameter branches: Delta_i^M = W_{i,M}^{local} - W_{r,M}; Delta_i^B = W_{i,B}^{local} - W_{r,B}; where Delta_i^M represents matrix branch difference data, W_{i,M}^{local} represents local matrix branch parameters, and W_{r,M} represents matrix parameters; Delta_i^B represents ordinary branch difference data, W_{i,B}^{local} represents local ordinary branch parameters, and W_{r,B} represents ordinary parameters. And Delta_i = Delta_i^M + Delta_i^B.

[0034] Furthermore, the matrix branch difference data is merged with the historical compressed residuals corresponding to the matrix branches in the local cache, and the ordinary branch difference data is merged with the historical compressed residuals corresponding to the ordinary branches in the local cache.

[0035] In one embodiment, if unified error feedback is enabled, the client merges the cached complete historical compression residuals into the current complete model difference data before compression, resulting in U_i=Delta_i+R_i; where Delta_i represents the complete model difference data and R_i represents the cached complete historical compression residuals.

[0036] Preferably, such as Figure 2 As shown, error feedback is performed on the two branch differences respectively. The client merges the historical compressed residual E_i^M of the matrix branch cache into the matrix branch difference data, resulting in U_i^M=Delta_i^M+E_i^M; and merges the historical compressed residual E_i^B of the ordinary branch cache into the ordinary branch difference data, resulting in U_i^B=Delta_i^B+E_i^B.

[0037] S103. Perform differential compression processing on the two merged branches to obtain a matrix branch compressed package and a normal branch compressed package. After merging, a complete compressed update package is obtained.

[0038] Specifically, for the merged matrix branch difference data, compression is performed by at least one of the following methods: low-rank decomposition compression, top-k sparsity compression according to matrix blocks, block sparsity compression, or matrix-aware quantization compression, to obtain the matrix branch compressed package. For the merged ordinary branch differential data, compression is performed using at least one of the following methods: b-bit fixed-point quantization compression, unified top-k sparsity compression, random sparsity compression, or entropy coding compression, to obtain the ordinary branch compressed package; wherein, the matrix branch and the ordinary branch are configured with compression strength parameters independently.

[0039] Furthermore, the matrix branch compressed package is merged and compressed with the ordinary branch compressed package to obtain a complete compressed update package.

[0040] In one embodiment, such as Figure 2 As shown, the client compresses the U_i^M and U_i^B obtained after merging and compressing the residuals, respectively, to obtain the matrix branch compressed package C_i^M and the ordinary branch compressed package C_i^B, which are then combined into a compressed update package C_i={C_i^M,C_i^B}. Compression methods include, but are not limited to: performing local top-k operations on matrix branches by matrix or tensor blocks, uploading low-rank factors after low-rank decomposition, matrix-aware quantization, or block sparsification; and performing fixed-point quantization, random quantization, unified top-k operations, random sparsity, or entropy coding on ordinary branches. Different branches can have their own compression configuration parameters rho_M and rho_B set.

[0041] Furthermore, the matrix branch compressed package and the ordinary branch compressed package are decompressed and reconstructed to obtain matrix branch reconstructed difference data and ordinary branch reconstructed difference data. The difference between the matrix branch difference data and the matrix branch reconstructed difference data is calculated and written as a new compressed residual into the buffer corresponding to the matrix branch, for use in residual merging in the next round of training. The difference between the ordinary branch difference data and the ordinary branch reconstructed difference data is calculated and written as a new compressed residual into the buffer corresponding to the ordinary branch, for use in residual merging in the next round of training.

[0042] In one alternative implementation, the client applies uniform top-k sparsity to the full model difference and writes the unuploaded coordinates to a residual buffer. This approach is simple to implement and suitable for lightweight CNNs or VGG11s federated experiments.

[0043] In another alternative implementation, the client uses low-rank or block compression for the Muon matrix branches and quantization compression for the ordinary parameter branches. This approach leverages the structural characteristics of matrix differences to improve expressive power under the same upload budget.

[0044] S104. Upload the complete compressed update package to the server to reconstruct the model differential data, complete the global model parameter update, and enter the next round of training.

[0045] Specifically, the complete compressed update package obtained from local training is aggregated to the server, and decompression and reconstruction are performed on the server to obtain the reconstructed model difference data corresponding to each client; and the reconstructed model difference data of each client is aggregated based on the preset aggregation rules to obtain the global reconstructed model difference data, the global model parameters are updated, and the next round of training begins; wherein, the preset aggregation rules are the FedAvg weighted average aggregation rule or the DiLoCo outer aggregation rule.

[0046] As a feasible implementation, the client only uploads compressed update packages. The server does not require receiving Muon momentum, orthogonalized intermediate matrices, or the client optimizer's internal state; it only receives compressed update packages containing matrix branch compressed packages and ordinary branch compressed packages. After decompression, the differential data is reconstructed, and then the global model is updated according to FedAvg weighted average or DiLoCo outer update rules. Simultaneously, the system records the uncompressed upload volume B_full, the compressed upload volume B_comp, and the compression ratio CR=B_full / B_comp to demonstrate bandwidth gains.

[0047] In one embodiment, the server receives compressed update packages from each client, performs low-rank reconstruction, dequantization, or sparse recovery on the matrix branches, and dequantization or sparse recovery on the ordinary branches, obtaining the matrix reconstruction difference hat{Delta}_i^M and the ordinary reconstruction difference hat{Delta}_i^B, which are then merged into an aggregate difference: hat{Delta}_i = hat{Delta}_i^M + hat{Delta}_i^B. Under the FedAvg interface, the server weights and merges the initial global model parameters with the aggregate differences from each client according to preset weights. Under the DiLoCo interface, the server or coordinating node uses hat{Delta}_i as the outer update input and updates the global model according to the DiLoCo outer optimization rules.

[0048] In one alternative implementation, the server maintains the FedAvg interface and adds a decompression module only at the receiving end; if the system uses DiLoCo, the decompressed aggregation update is handed over to the outer optimizer for processing, without changing the client's local Muon training process.

[0049] In another alternative implementation, the client dynamically adjusts the Muon activation layer, orthogonalization frequency, compression ratio, and residual buffer precision based on local computing power and network conditions. For example, a client with weak computing power may only enable Muon for larger linear layers or attention projection matrices, and increase the compression ratio when the network is congested.

[0050] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes, substitutions, combinations, or equivalent implementations made by those skilled in the art based on the present invention, such as replacing the specific compressor, replacing the approximate orthogonalization algorithm, or adjusting the FedAvg or DiLoCo outer aggregator, are all within the scope of protection claimed by the present invention.

[0051] The present invention aims to solve the following technical problems in existing bandwidth-constrained federated training: (1) The complete model difference generated by the local update of the Muon matrix still needs to be uploaded in high bytes, which is difficult to adapt to the uplink constraints of vehicle, roadside and regional edge nodes; (2) Although ordinary compression FedAvg or DiLoCo can reduce the amount of upload, the update quality before compression is insufficient, which may weaken the convergence effect of the perception model; (3) The difference structure of matrix branch and ordinary parameter branch is different. The existing scheme lacks a system mechanism for separate compression, separate maintenance of error feedback and unified decompression and aggregation on the server side; (4) The low communication benefit lacks a unified calculation caliber, which is not conducive to proving the technical effect with uncompressed upload amount, compressed upload amount and compression rate.

[0052] Based on this, the present invention proposes a low-communication federated training method and system that combines Muon or approximately orthogonalized local updates with communication compression and error feedback. Figure 3 This is a schematic diagram of Muon branch differential compression data interaction for an autonomous driving perception model provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the server or regional coordination node distributes the global perception model in each training round. Vehicle-side, roadside, or regional nodes execute dual-branch training locally, consisting of a Muon matrix branch and a backup optimizer branch. The two-dimensional matrix parameters are updated using Muon or approximate orthogonalization, while biases, normalized layers, one-dimensional parameters, and parameters unsuitable for orthogonalization are updated using the AdamW optimizer or the SGD backup optimizer. After completing local training, the client does not directly upload the complete difference data. Instead, it splits the model difference data generated by the Muon matrix update into matrix branch difference data and ordinary branch difference data, compressing and maintaining separate error feedback caches for each. The server receives the compressed update package, decompresses it, performs inverse quantization or low-rank reconstruction, and then aggregates the difference data according to the FedAvg or DiLoCo outer interface.

[0053] The following describes this solution in detail through four specific application scenario examples: Example 1: Low-bandwidth upload training of vehicle-road cooperative autonomous driving perception models: Multiple vehicle-end devices, roadside units, and regional edge nodes wish to jointly train models for road target recognition, traffic participant detection, abnormal event recognition, or multimodal perception. However, the original video, radar point cloud, location trajectory, and locally labeled data cannot leave the acquisition nodes, and the uplink bandwidth of edge nodes is limited. Ordinary FedAvg-AdamW requires uploading the complete model difference, resulting in high communication volume; directly using compressed FedAvg may affect convergence due to insufficient quality of local update directions before compression.

[0054] After applying this invention, each vehicle-side, roadside, or regional node updates the convolutional matrix, linear layer, and attention projection matrix using Muon or approximately orthogonalization during local training, and uses AdamW or SGD for the bias and normalization layers. After training, the node splits the difference generated by the Muon matrix update into matrix branch difference and ordinary branch difference, and performs low-rank, block top-k, quantization, or sparse compression on each branch, retaining the compressed residuals through branch error feedback caching. After decompression by the server, FedAvg aggregation is continued, thereby striving for higher validation accuracy or lower validation loss under the same communication budget.

[0055] Example 2: Low-Frequency Synchronization Training with DiLoCo Across Regions: Multiple regional worker nodes wish to train large visual or multimodal models. Each node possesses local data and computing resources, but cross-regional synchronization is costly. DiLoCo can reduce the synchronization frequency, but the model updates uploaded during synchronization may still be substantial, and local multi-step training demands higher update quality.

[0056] After applying this invention, each working node enables the matrix parameter Muon branch and the ordinary parameter backup optimizer branch in the inner layer local training of DiLoCo; before outer layer synchronization, the model update is compressed in low rank, in blocks, or with quantization, and the branch error feedback cache is used to track unuploaded errors. The server side or coordinating node only needs to reconstruct the compressed update into the outer layer update vector, and then use the DiLoCo aggregation interface. This method is suitable for low-frequency synchronization training across data centers, availability zones, or wide area networks.

[0057] Example 3: Upload Budget Control for Edge Natural Language or Multimodal Federated Fine-tuning: When multiple mobile terminals, enterprise private corpus nodes, or multimodal edge devices participate in federated fine-tuning, the network quality, battery status, and privacy constraints of different clients may vary. The system can select the compression rate based on the client's upload budget, enabling a more complete Muon matrix branch for clients with stronger computing power, and reducing the orthogonalization frequency or enabling Muon only for key matrix layers for resource-constrained clients. This approach facilitates balancing local computation, upload communication, and global model performance in edge text, multimodal, and personalized model training.

[0058] Example 4: Optional Application of Cross-Institutional Federated Training of Medical Images: When jointly training image classification, segmentation, or assisted diagnostic models in hospitals or medical imaging centers, the branch differential compression upload mechanism of this invention can also be used. In this scenario, the client can correspond to a hospital server or imaging center node. The matrix branch contains convolutional layers, linear layers, or visual Transformer projection matrices, while the ordinary branch contains bias and normalization parameters. This scenario is an optional application; the primary implementation scenario of this invention remains the low-bandwidth upload training of vehicle-road cooperative autonomous driving perception models.

[0059] In addition, embodiments of the present invention also provide a low-communication federated training system for autonomous driving perception models, such as... Figure 4 As shown, the low-communication federated training system 400 for autonomous driving perception models specifically includes: The dual-branch training module 410 is used to perform dual-branch training on the global model parameters of this round issued by the server based on local data, and generate complete model difference data; wherein, the dual-branch training includes: training the matrix parameters through the Muon approximate orthogonalization optimizer and training the ordinary parameters through the backup optimizer; The compression update module 420 is used to split the complete model difference data into matrix branch difference data and ordinary branch difference data, and merge them with the corresponding historical compression residuals respectively; perform differential compression processing on the two branch difference data after merging to obtain matrix branch compressed package and ordinary branch compressed package, and merge them to obtain complete compressed update package. The aggregation update module 430 is used to upload the complete compressed update package to the server for model differential data reconstruction, complete the global model parameter update, and enter the next round of training.

[0060] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0061] The foregoing has described specific embodiments of the present invention. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0062] The above description is merely an embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the embodiments of the present invention should be included within the protection scope of the present invention.

Claims

1. A low-communication federated training method for perception models in autonomous driving, characterized in that, The method includes: Based on local data, the global model parameters for this round issued by the server are trained in a two-branch manner to generate complete model difference data; wherein, the two-branch training includes: training the matrix parameters through the Muon approximate orthogonalization optimizer and training the ordinary parameters through the backup optimizer; The complete model difference data is split into matrix branch difference data and ordinary branch difference data, and then merged with the corresponding historical compressed residuals. The merged two branch difference data are subjected to differential compression processing to obtain matrix branch compressed package and ordinary branch compressed package, which are then merged to obtain complete compressed update package. The complete compressed update package is uploaded to the server for model differential data reconstruction, completing the global model parameter update and entering the next round of training.

2. The low-communication federated training method for an autonomous driving perception model according to claim 1, characterized in that, Before performing dual-branch training on the global model parameters issued by the server based on local data to obtain complete model difference data, the method further includes: Receive the global model parameters for this round from the server; wherein, the global model parameters for this round are the global model parameters for the current training round selected by the server after initializing the autonomous driving perception model; Based on the structural properties of the global model parameters in this round, they are divided into matrix parameters and ordinary parameters.

3. A low-communication federated training method for an autonomous driving perception model according to claim 2, characterized in that, Based on the structural properties of the global model parameters in this round, they are divided into matrix parameters and ordinary parameters, specifically including: The two-dimensional matrix after convolution kernel rearrangement, the linear layer weight matrix, the attention projection matrix, and the feedforward network weight matrix in the current round of global model parameters are divided into matrix parameters. Bias parameters, normalization layer parameters, one-dimensional embedding scaling parameters, and small-size parameters that cannot be orthogonalized are classified as ordinary parameters.

4. The low-communication federated training method for an autonomous driving perception model according to claim 1, characterized in that, Based on local data, the global model parameters for this round, issued by the server, are trained in a dual-branch manner to generate complete model difference data, specifically including: Based on local data, several training steps are performed on the matrix parameters and ordinary parameters in the current round of global model parameters to obtain the local model parameters; The difference between the local model parameters and the global model parameters in this round is calculated to obtain the complete model difference data.

5. A low-communication federated training method for an autonomous driving perception model according to claim 1, characterized in that, The matrix parameters are trained using the Muon approximate orthogonalization optimizer, specifically including: The gradient matrix of the matrix parameters in the current training step is calculated using the Muon approximate orthogonalization optimizer, and the current momentum matrix is ​​updated based on the gradient matrix and the preset momentum coefficients. Construct an estimated update direction matrix based on the gradient matrix and the updated current momentum matrix; The estimated update direction matrix is ​​approximately orthogonalized to obtain the orthogonalized matrix update direction. The matrix parameters are updated based on the weight decay coefficient, the matrix branch-specific learning rate, and the update direction of the orthogonalized matrix, and then the training proceeds to the next step.

6. A low-communication federated training method for an autonomous driving perception model according to claim 4, characterized in that, The complete model difference data is split into matrix branch difference data and ordinary branch difference data, and then merged with the corresponding historical compressed residuals, specifically including: Obtain the local matrix branch parameters corresponding to the matrix parameters and the local ordinary branch parameters corresponding to the ordinary parameters from the local model parameters; Calculate the difference between the local matrix branch parameter and the matrix parameter to obtain the matrix branch difference data; calculate the difference between the local ordinary branch parameter and the ordinary parameter to obtain the ordinary branch difference data; The matrix branch differential data is merged with the historical compressed residuals corresponding to the matrix branches in the local cache, and the ordinary branch differential data is merged with the historical compressed residuals corresponding to the ordinary branches in the local cache.

7. A low-communication federated training method for an autonomous driving perception model according to claim 1, characterized in that, The differential data from the two merged branches are subjected to differential compression processing to obtain a matrix branch compressed package and a normal branch compressed package. After merging, a complete compressed update package is obtained, which specifically includes: The merged matrix branch difference data is compressed using at least one of the following methods: low-rank decomposition compression, top-k sparsification compression according to matrix blocks, block sparsification compression, or matrix-aware quantization compression, to obtain the matrix branch compressed package. For the merged ordinary branch differential data, compression is performed using at least one of the following methods: b-bit fixed-point quantization compression, unified top-k sparsity compression, random sparsity compression, or entropy coding compression, to obtain the ordinary branch compressed package; wherein, the matrix branch and the ordinary branch are configured with compression strength parameters independently. The matrix branch compressed package is merged and compressed with the ordinary branch compressed package to obtain the complete compressed update package.

8. A low-communication federated training method for an autonomous driving perception model according to claim 1, characterized in that, After performing differential compression on the merged two branch differential data to obtain matrix branch compressed packages and ordinary branch compressed packages, the method further includes: The matrix branch compressed package and the ordinary branch compressed package are decompressed and reconstructed to obtain matrix branch reconstructed differential data and ordinary branch reconstructed differential data; The difference between the matrix branch difference data and the matrix branch reconstructed difference data is calculated and written as a new compressed residual into the cache area corresponding to the matrix branch for residual merging in the next round of training. The difference between the difference data of the ordinary branch and the difference data of the reconstructed ordinary branch is calculated and written into the cache area corresponding to the ordinary branch as a new compressed residual for the next round of training residual merging.

9. A low-communication federated training method for an autonomous driving perception model according to claim 1, characterized in that, The complete compressed update package is uploaded to the server for model difference data reconstruction, completing the global model parameter update, specifically including: The complete compressed update package obtained from local training is aggregated to the server, where decompression and reconstruction are performed to obtain the reconstructed model difference data corresponding to each client. The reconstructed model difference data of each client is then aggregated based on a preset aggregation rule to obtain global reconstructed model difference data. The global model parameters are then updated, and the next round of training begins. The preset aggregation rule is either the FedAvg weighted average aggregation rule or the DiLoCo outer aggregation rule.

10. A low-communication federated training system for autonomous driving perception models, characterized in that, The system includes: The dual-branch training module is used to perform dual-branch training on the global model parameters issued by the server based on local data, generating complete model difference data; wherein, the dual-branch training includes: training the matrix parameters through the Muon approximate orthogonalization optimizer, and training the ordinary parameters through the backup optimizer; The compression and update module is used to split the complete model difference data into matrix branch difference data and ordinary branch difference data, and merge them with the corresponding historical compression residuals respectively; perform differential compression processing on the two branch difference data after merging to obtain matrix branch compressed package and ordinary branch compressed package, and merge them to obtain complete compressed update package; The aggregation update module is used to upload the complete compressed update package to the server for model differential data reconstruction, complete the global model parameter update, and enter the next round of training.

Images