Efficient federated learning framework based on structured pruning and knowledge distillation

Abstract

The invention discloses an Internet of Vehicles federated learning system based on structured pruning and knowledge distillation, which is suitable for resource-constrained environments of the Internet of Vehicles and the Internet of Things. Firstly, an intelligent vehicle selection algorithm is designed at a road side unit (RSU) end and used for selecting representative vehicle subsets to participate in model training in each round of training. Secondly, in a global model distribution stage, a lightweight personalized sub-model is generated through a structured pruning mechanism so as to adapt to calculation and storage heterogeneity of different vehicles; thirdly, in the local training process, a knowledge distillation strategy based on a global model is introduced, so that the influence of non-independent identically distributed (Non-IID) data on the convergence performance of the model is relieved; then, aiming at the problem of instability caused by pruning model aggregation, a recovery model aggregation (RMA) method is proposed, and stable convergence of a global model is realized; and finally, in the model transmission stage, model parameters are further compressed through a quantization technology, and the communication overhead is remarkably reduced.

Description

Technical Field

[0001] This invention relates to federated learning (FL) technology in vehicle-to-everything (V2X) and Internet of Things (IoT) environments, specifically to a method for efficient, low-overhead, and privacy-preserving distributed model training in resource-constrained networks. This method reduces communication and storage overhead by selecting representative vehicles for training at edge units (RSUs), performing structured pruning on the global model to generate a lightweight quantum model, mitigating the impact of non-independent and identically distributed (Non-IID) data through knowledge distillation, and introducing restorative model aggregation (RMA) and quantization compression parameters. Background Technology

[0002] In recent years, the rapid development of communication technologies, embedded sensors, and artificial intelligence has made the construction of in-vehicle networks in modern automobiles possible, becoming a key component of Intelligent Transportation Systems (ITS). These networks include vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and more broadly vehicle-to-everything (V2X) communications, aiming to improve traffic safety, optimize transportation efficiency, and drive the development of autonomous driving technologies. As vehicle intelligence and connectivity increase, vehicles continuously generate and exchange vast amounts of data, including sensor information, driving behavior, and environmental perception data. This data plays a crucial role in applications such as real-time traffic monitoring, inter-vehicle cooperative perception, predictive control of intelligent transportation systems, and early warning of potentially dangerous driving behaviors.

[0003] However, the collection and utilization of large-scale distributed data also brings significant challenges, including data privacy protection, communication overhead, and computational efficiency. Traditional centralized machine learning methods typically require aggregating data from all vehicles and infrastructure to a central server, which not only poses serious privacy risks but also increases the burden on communication networks and cloud computing resources. To address these issues, Federated Learning (FL) has been proposed, enabling collaborative model training among distributed clients (such as vehicles or roadside units) without directly sharing raw data. Within the FL framework, each client uses its own private data for local model training, only uploading model updates (such as gradients or weights) to the central server for aggregation. This decentralized training approach effectively protects data privacy, reduces communication overhead, and improves system scalability, making it particularly suitable for resource-constrained vehicle-to-everything (V2X) environments. With FL, intelligent transportation systems can leverage the collective intelligence of vehicles and roadside infrastructure to build robust, adaptive models to support various tasks such as traffic flow prediction, driving behavior modeling, object detection, and collaborative decision-making, while ensuring data privacy, reducing communication overhead, and improving overall system efficiency.

[0004] Against this backdrop, in-depth research into how to build efficient, privacy-preserving, and resource-aware federated learning systems in the context of connected vehicles and the Internet of Things (IoT) is particularly urgent and necessary. This will not only help to fully utilize the data resources of distributed vehicles and edge devices and improve the decision-making capabilities of intelligent transportation systems and IoT applications, but also has significant practical implications for promoting smart transportation, autonomous driving, and the digital transformation and intelligent upgrading of various industries. Summary of the Invention

[0005] This invention addresses the model training optimization problem in resource-constrained vehicle-to-everything (V2X) environments by proposing an efficient federated learning system for V2X that combines structured pruning and knowledge distillation.

[0006] This invention is achieved using the following technical solution:

[0007] A network of vehicles federated learning system based on structured pruning and knowledge distillation, characterized by the following steps:

[0008] 1) Establish an edge-to-edge federated learning framework for vehicle-to-everything (V2X) communication, deploying the model in vehicles and Roadside Units (RSUs). The vehicle-side is responsible for training the model using local data, while the RSUs are responsible for aggregating the global model. This ensures that the raw perception data does not leave the vehicle.

[0009] 2) Perform vehicle selection and structured pruning operations on the RSU side to generate personalized lightweight sub-models adapted to the heterogeneous computing power of vehicles, and use quantization technology to further compress parameter data during model transmission to reduce communication overhead and improve transmission efficiency.

[0010] 3) Train the compressed model locally on the vehicle based on the global model distillation loss function and upload the updated model.

[0011] 4) Perform model recovery and aggregation update at the RSU end to obtain a new global model until the convergence condition is met.

[0012] In the above technical solution, the edge-to-end federated learning framework for the Internet of Vehicles described in step 1) further comprises four parts:

[0013] (1) Status monitoring module: RSU collects data such as vehicle position, speed, and computing power at the beginning of each round of federated learning in order to select reliable vehicles to participate in federated learning.

[0014] (2) Model Compression and Model Distribution Module: Based on the selected vehicle status information, RSU designs suitable sub-models for each vehicle using structured pruning techniques. The compressed sub-models are then distributed to the selected vehicles.

[0015] (3) Local training and model upload module: Selected vehicles utilize local data Dn Distillation training is performed on the sub-model along with the global model. After the model training is complete, the trained sub-model is uploaded.

[0016] (4) Model Recovery Aggregation Module: RSU collects the sub-models after training. The sub-model structure is recovered using the model recovery module, and then all recovered sub-models are aggregated using federated weighted average to form a new global model.

[0017] Furthermore, the vehicle-to-everything (V2X) federated learning system based on structured pruning and knowledge distillation described in step 2) is characterized by the following Greedy Vehicle Selection Based on Importance Score algorithm used by the RSU to select vehicles to participate in each round of federated learning:

[0018] This algorithm, based on a vehicle importance scoring mechanism, aims to select the most representative and reliable vehicles for federated learning training in each round of communication, thereby improving the efficiency and performance of global model aggregation. The algorithm takes vehicle bandwidth, computing power, channel conditions, upload success rate, and their corresponding weights as input, comprehensively considering the allocation of communication and computing resources. It calculates and ranks the importance scores of each vehicle, ultimately selecting the optimal set of vehicles as output, thus achieving efficient vehicle selection and model updating under resource-constrained conditions. The specific process includes:

[0019] ① Information Collection Phase: RSU collects information on all vehicles participating in the Federal Learning program. D n V n , B n h n,k Data such as...

[0020] ②Importance parameter calculation stage: The same data collected from each vehicle are normalized, and the importance score is calculated by multiplying the normalization results.

[0021] ③ Vehicle selection stage: Filter out vehicles whose importance scores do not reach the threshold, and select the remaining vehicles using a greedy algorithm.

[0022] Furthermore, the Grouped Federated Model Aggregation and Recovery algorithm for RSU model recovery aggregation described in step 4) is as follows:

[0023] Based on a federated learning framework, model pruning techniques, and the principle of grouping and aggregating by pruning rate, this method aims to efficiently aggregate sub-models trained on different vehicles and recover a complete and updated global model to address the heterogeneity of clients in a federated network. The specific process includes:

[0024] ① Sub-model collection phase: RSU collects all sub-models of the selected vehicles after local training is completed.

[0025] ② Sub-model recovery phase: RSU uses the pruned indexes of the sub-models to restore the structure of the sub-models to the global model structure.

[0026] ③ Global Model Aggregation Phase: RSU aggregates the sub-models of the response structure into a new round of global model by using a federated weighted average.

[0027] Furthermore, the FedPKD system is as follows:

[0028] Based on the principles of federated learning, personalized knowledge distillation, model pruning, and quantization, this algorithm aims to efficiently train a global model while addressing the computational and communication heterogeneity of vehicle users. The algorithm distributes the global model and pruned sub-models to each vehicle, trains them locally using knowledge distillation, and finally reconstructs the aggregated models uploaded by each vehicle.

[0029] Furthermore, the objective function of the vehicle-to-everything (V2X) federated learning system based on structured pruning and knowledge distillation described in step 4) is:

[0030] The problem of minimizing the energy consumption and model transmission latency for local model training in vehicles can be expressed as:

[0031]

[0032] in,

[0033] 1) This represents the pruning rate of the model, M is the number of vehicles selected to participate in federated learning in each round, η is the weighting coefficient, and T... k It is the total latency of all vehicles participating in federated learning in round k, E k It is the total energy consumption of all vehicles participating in federated learning in round k, T k The calculation formula is as follows:

[0034]

[0035] 2) This is the time required for vehicle n to complete parameter training and parameter transmission in the kth round. H is the time required for the k-th round of RSU receiving the model and model aggregation. n It represents the number of times the local model of vehicle n is iterated during training.

[0036] When a vehicle is selected to participate in the k-th round of federated learning, it first receives the corresponding sub-model parameters from the RSU and performs multiple rounds of iterative updates using local data. This represents the time required for the vehicle to train the model locally for each round. After local training is complete, the vehicle uploads the updated model parameters to the RSU. The time required to transmit vehicle model parameters to the RSU; after receiving sub-models uploaded by multiple vehicles, the RSU completes model recovery and global aggregation. This is the time required to receive the sub-model trained on the vehicle side. This is the time required to aggregate the RSU model parameters. The aggregated global model undergoes adaptive structured pruning to improve model lightweighting and communication efficiency. Finally, the pruned global model is redistributed to the vehicle nodes. The above latency calculation formula is as follows:

[0037]

[0038] Among them, D n This represents the amount of local data held by the nth vehicle; This refers to the vehicle's GPU computing power; Let S represent the model compression ratio of vehicle n in the k-th round, and S be the model size. B is the uplink transmission rate of the node; P is the bandwidth. n,k For transmit power and channel gain h n,k Distance d between vehicle and RSU n,k Path loss index The transmission performance of the channel is determined by both the noise power γ and the noise power γ. For edge servers, Here, M represents the server's uplink receiving rate, and M represents the number of vehicles participating in the upload; furthermore, This represents the computational cost required to aggregate the various vehicle models. 3) The total energy consumption E of all vehicles participating in federated learning in the kth round. k It mainly consists of the following three parts: energy consumption for local model training on the vehicle side. Energy consumption during model parameter upload and energy consumption of RSU receiving vehicle model parameters. And the computational energy consumption of performing global model aggregation at the RSU end. E k The calculation formula is as follows:

[0039]

[0040]

[0041] in, This represents the energy consumption of the m-th vehicle node during the local model training process in the k-th round; This indicates the communication energy consumption of the RSU when receiving model parameters uploaded by each vehicle; Communication energy consumption when uploading model parameters for the m-th vehicle node; α r c is the power factor of the RSU. r This indicates the number of GPU cycles required for each unit of data to be processed by the RSU task. This refers to the processing frequency of the RSU; while S... n This represents the scale of the model aggregation task related to the nth vehicle. The formula for calculating the energy consumption mentioned above is as follows:

[0042]

[0043] Where α n This represents the calculated power factor of vehicle n. The number of GPU cycles required per unit of data for the task. D represents the local computation frequency of vehicle n. n H represents the amount of data that vehicle n needs to process. n P represents the number of local training iterations for vehicle n; n,k Let n be the transmission power of vehicle n during parameter transmission; P represents the parameter transfer time of RSU in task k. n,k This represents the transmission power of the RSU during parameter transmission.

[0044] The present invention also provides a process for performing federated training using the above method, as detailed below:

[0045] 1) Structured Pruning: RSU effectively removes redundant parameters in the model by adopting a structured pruning strategy, thereby reducing the computational overhead required for model training and significantly improving the efficiency of model parameters during transmission.

[0046] 2) Quantization: During the transmission of model parameters, RSU uses quantization technology to convert high-precision parameters into low-precision representations, thereby significantly reducing the amount of data transmitted and improving communication efficiency.

[0047] 3) Knowledge Distillation: When training the local model on the vehicle, a knowledge distillation mechanism is introduced to alleviate the performance difference caused by uneven distribution of local data (non-IID), thereby significantly improving the convergence speed and stability of the global model.

[0048] 4) Model Recovery: During the model aggregation phase, since the sub-model structures uploaded by each vehicle may differ, RSU first restores the sub-models with different structures to a unified global model structure, and then performs the federated aggregation process to ensure the consistency of model parameters and the reliability of aggregation results.

[0049] 5) Repeat steps 1), 2), 3) and 4) until the accuracy of the global model reaches the target value while minimizing latency and energy consumption.

[0050] The inventive principle of this invention:

[0051] This invention addresses the optimization problem of federated learning model training in resource-constrained vehicle-to-everything (V2X) and Internet of Things (IoT) environments. It proposes an efficient federated learning system to reduce the computational, storage, and communication overhead of vehicles and edge units while ensuring rapid and stable convergence of the global model. The system introduces an intelligent vehicle selection algorithm at the roadside unit (RSU) level to select representative vehicles; it generates lightweight, personalized sub-models through structured pruning to adapt to vehicle heterogeneity; it mitigates the impact of non-IID data through local training combined with knowledge distillation; and it further optimizes aggregation stability and communication efficiency through restorative model aggregation (RMA) and quantization compression. This system achieves efficient, stable, and reliable federated learning in resource-constrained environments.

[0052] The beneficial effects of this invention are as follows:

[0053] This paper proposes a novel, highly efficient federated learning system for vehicular networks (V2V) based on structured pruning and knowledge distillation. This method effectively reduces the computational, storage, and communication overhead of vehicles and edge units during model training, while ensuring rapid and stable convergence of the global model. The system achieves efficient, stable, and reliable federated learning in resource-constrained V2V and IoT environments through the organic combination of intelligent vehicle selection, lightweight sub-model generation, knowledge distillation strategies, recapitulation model aggregation (RMA), and quantization compression. This significantly improves model training efficiency and resource utilization. Attached Figure Description

[0054] Figure 1 A research motivation diagram for federated learning in vehicle networks;

[0055] Figure 2 This is a diagram of the FedPKD system framework.

[0056] Figure 3 A graph showing the test accuracy versus training loss analysis of driver distraction detection datasets based on different federated learning methods;

[0057] Figure 4 A graph showing the test accuracy and training loss analysis of different federated learning methods on the CIFAR-10 dataset;

[0058] Figure 5 The energy costs of different methods to achieve the target accuracy;

[0059] Figure 6 Different methods have varying bandwidth overhead to achieve the target accuracy;

[0060] Figure 7 This is a data distribution diagram of vehicles under non-independent and identically distributed conditions;

[0061] Figure 8 The time required for a vehicle to achieve the target accuracy under different non-independent and identically distributed levels; Detailed Implementation

[0062] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0063] A network of vehicles federated learning system based on structured pruning and knowledge distillation, characterized by the following steps:

[0064] 1) Establish an edge-to-edge federated learning framework for vehicle-to-everything (V2X) communication, deploying the model in vehicles and Roadside Units (RSUs). The vehicle-side is responsible for training the model using local data, while the RSUs are responsible for aggregating the global model. This ensures that the raw perception data does not leave the vehicle.

[0065] 2) Perform vehicle selection and structured pruning operations on the RSU side to generate personalized lightweight sub-models adapted to the heterogeneous computing power of vehicles, and use quantization technology to further compress parameter data during model transmission to reduce communication overhead and improve transmission efficiency.

[0066] 3) Train the compressed model locally on the vehicle based on the global model distillation loss function and upload the updated model.

[0067] 4) Perform model recovery and aggregation update at the RSU end to obtain a new global model until the convergence condition is met.

[0068] The vehicular network federated learning system proposed in this invention, based on structured pruning and knowledge distillation, is as follows: Figure 1 The diagram mainly includes four parts.

[0069] (1) Status monitoring module: RSU collects data such as vehicle position, speed, and computing power at the beginning of each round of federated learning in order to select reliable vehicles to participate in federated learning.

[0070] (2) Model Compression and Model Distribution Module: Based on the selected vehicle status information, RSU designs suitable sub-models for each vehicle using structured pruning techniques. The compressed sub-models are then distributed to the selected vehicles.

[0071] (3) Local training and model upload module: Selected vehicles utilize local data D n Distillation training is performed on the sub-model along with the global model. After the model training is complete, the trained sub-model is uploaded.

[0072] (4) Model Recovery Aggregation Module: RSU collects the sub-models after training. The sub-model structure is recovered using the model recovery module, and then all recovered sub-models are aggregated using federated weighted average to form a new global model.

[0073] Figure 2 The system framework diagram of FedPKD is presented. The system is initiated by Roadside Units (RSUs). In each training round, the RSU selects representative vehicles and performs l1-norm structured pruning on the current global model, generating a customized lightweight quantum model for each selected vehicle. Subsequently, the RSU distributes the global model (teacher) and corresponding sub-models (students) to the vehicles. After receiving the model, the vehicles train their sub-models using local data. By introducing a knowledge distillation mechanism, the system minimizes the cross-entropy loss while minimizing the difference from the global model's output, thus learning the knowledge of the global model and mitigating the non-independent and identically distributed problem. After training, the vehicles upload updated local sub-models to the RSU. After collecting and restoring the model structure, the RSU aggregates the parameters from each vehicle to generate a new global model, achieving continuous model optimization and performance improvement.

[0074] The problem of minimizing the energy consumption and model transmission latency for local model training in vehicles can be expressed as:

[0075]

[0076] in,

[0077] 1) This represents the pruning rate of the model, M is the number of vehicles selected to participate in federated learning in each round, η is the weighting coefficient, and T... k It is the total latency of all vehicles participating in federated learning in round k, E k It is the total energy consumption of all vehicles participating in federated learning in round k, T k The calculation formula is as follows:

[0078]

[0079] 2) This is the time required for vehicle n to complete parameter training and parameter transmission in the kth round. H is the time required for the k-th round of RSU receiving the model and model aggregation. n It represents the number of times the local model of vehicle n is iterated during training.

[0080] When a vehicle is selected to participate in the k-th round of federated learning, it first receives the corresponding sub-model parameters from the RSU and performs multiple rounds of iterative updates using local data. This represents the time required for the vehicle to train the model locally for each round. After local training is complete, the vehicle uploads the updated model parameters to the RSU. The time required to transmit vehicle model parameters to the RSU; after receiving sub-models uploaded by multiple vehicles, the RSU completes model recovery and global aggregation. This is the time required to receive the sub-model trained on the vehicle side. This is the time required to aggregate the RSU model parameters. The aggregated global model undergoes adaptive structured pruning to improve model lightweighting and communication efficiency. Finally, the pruned global model is redistributed to the vehicle nodes. The above latency calculation formula is as follows:

[0081]

[0082] Among them, D n This represents the amount of local data held by the nth vehicle; This refers to the vehicle's GPU computing power; Let S represent the model compression ratio of vehicle n in the k-th round, and S be the model size. B is the uplink transmission rate of the node; P is the bandwidth. n,k For transmit power and channel gain h n,k Distance d between vehicle and RSU n,k Path loss index The transmission performance of the channel is determined by both the noise power γ and the noise power γ. For edge servers, Here, M represents the server's uplink receiving rate, and M represents the number of vehicles participating in the upload; furthermore, This represents the computational cost required to aggregate the various vehicle models. 3) The total energy consumption E of all vehicles participating in federated learning in the kth round. k It mainly consists of the following three parts: energy consumption for local model training on the vehicle side. Energy consumption during model parameter upload and energy consumption of RSU receiving vehicle model parameters. And the computational energy consumption of performing global model aggregation at the RSU end. E k The calculation formula is as follows:

[0083]

[0084] in, This represents the energy consumption of the m-th vehicle node during the local model training process in the k-th round; This indicates the communication energy consumption of the RSU when receiving model parameters uploaded by each vehicle; Communication energy consumption when uploading model parameters for the m-th vehicle node; α r c is the power factor of the RSU. r This indicates the number of GPU cycles required for each unit of data to be processed by the RSU task. This refers to the processing frequency of the RSU; while S... n This represents the scale of the model aggregation task related to the nth vehicle. The formula for calculating the energy consumption mentioned above is as follows:

[0085]

[0086] Where α n This represents the calculated power factor of vehicle n. The number of GPU cycles required per unit of data for the task. D represents the local computation frequency of vehicle n. n H represents the amount of data that vehicle n needs to process. n P represents the number of local training iterations for vehicle n; n,k Let n be the transmission power of vehicle n during parameter transmission; P represents the parameter transfer time of RSU in task k. n,k This represents the transmission power of the RSU during parameter transmission.

[0087] The present invention also provides a process for performing federated training using the above method, as detailed below:

[0088] 1) Structured Pruning: RSU effectively removes redundant parameters in the model by adopting a structured pruning strategy, thereby reducing the computational overhead required for model training and significantly improving the efficiency of model parameters during transmission.

[0089] 2) Quantization: During the transmission of model parameters, RSU uses quantization technology to convert high-precision parameters into low-precision representations, thereby significantly reducing the amount of data transmitted and improving communication efficiency.

[0090] 3) Knowledge Distillation: When training the local model on the vehicle, a knowledge distillation mechanism is introduced to alleviate the performance difference caused by uneven distribution of local data (non-IID), thereby significantly improving the convergence speed and stability of the global model.

[0091] 4) Model Recovery: During the model aggregation phase, since the sub-model structures uploaded by each vehicle may differ, RSU first restores the sub-models with different structures to a unified global model structure, and then performs the federated aggregation process to ensure the consistency of model parameters and the reliability of aggregation results.

[0092] Repeat steps 1), 2), 3) and 4) until the accuracy of the global model reaches the target value while minimizing latency and energy consumption.

[0093] Simulation results

[0094] This invention's method (FedPKD) is compared with three different conventional algorithms: FedAvg, FedPorx, and Moon. Specifically, the test accuracy, energy consumption, and bandwidth consumption after each round of global model training are compared.

[0095] Figure 3 and Figure 4 This demonstrates that FedPKD achieved the fastest accuracy improvement in the early stages of training on two different datasets. Simultaneously, FedPKD also performed best in terms of training loss. These results demonstrate the significant advantages of FedPKD in convergence speed, final performance, and robustness.

[0096] By comparison Figure 5 and Figure 6 It can be observed that FedPKD exhibits significant advantages over other algorithms in terms of bandwidth usage and energy consumption during model training. This indicates that while achieving efficient communication and energy consumption control, the algorithm can effectively reduce resource consumption during training, thus better meeting the dual requirements of real-time performance and energy efficiency in connected vehicle scenarios.

[0097] from Figure 7 It can be seen that the simulation environment constructed in this study fully considers the uneven distribution of real vehicle data and adopts a representative non-IID scenario to verify the robustness and generalization ability of the algorithm in complex environments.

[0098] Furthermore, from Figure 8 The results show that FedPKD outperforms other algorithms in various non-IID scenarios. This demonstrates that FedPKD not only excels in model convergence stability but also achieves a good balance between energy consumption and performance, proving its high application potential and practical value in real-world in-vehicle federated learning systems.

Claims

1. A connected vehicle federated learning system based on structured pruning and knowledge distillation, characterized in that, Includes the following steps: 1) Establish an edge-to-edge federated learning framework for vehicle-to-everything (V2X) networks, deploying the model in vehicles and roadside units (RSUs). The vehicle side is responsible for training the model using local data, while the RSU side is responsible for aggregating the global model to ensure that the original perception data does not leave the vehicle side. 2) Perform vehicle selection and structured pruning operations on the RSU side to generate personalized lightweight sub-models adapted to the heterogeneous computing power of vehicles, and use quantization technology to further compress parameter data during model transmission to reduce communication overhead and improve transmission efficiency. 3) Train the compressed model locally on the vehicle based on the global model distillation loss function and upload the updated compressed model; 4) Perform model recovery and aggregation update at the RSU end to obtain a new global model until the convergence condition is met.

2. The vehicle network federated learning system based on structured pruning and knowledge distillation as described in claim 1, characterized in that, The edge-to-edge federated learning framework for vehicle-to-everything (V2X) communication described in step 1) consists of four parts: (1) Status monitoring module: RSU collects data such as vehicle position, speed, and computing power at the beginning of each round of federated learning in order to select reliable vehicles to participate in federated learning; (2) Model compression and model distribution module: Based on the selected vehicle status information, RSU uses structured pruning technology to design a suitable sub-model for each vehicle and distributes the compressed sub-model to the selected vehicle. (3) Local training and model upload module: Selected vehicles utilize local data D n Distillation training of sub-models with the global model, and upload the trained sub-models after the model training is completed; (4) Model recovery aggregation module: RSU collects the sub-models after training, uses the model recovery module to recover the sub-model structure, and then uses federated weighted average to aggregate all recovered sub-models to form a new round of global model.

3. The vehicle network federated learning system based on structured pruning and knowledge distillation as described in claim 2, characterized in that, Step 2) describes the Greedy Vehicle Selection Based on Importance Score algorithm for RSU to select vehicles to participate in each round of federated learning, as follows: This algorithm, based on a vehicle importance scoring mechanism, aims to select the most representative and reliable vehicles for federated learning training in each round of communication, thereby improving the efficiency and performance of global model aggregation. The algorithm takes vehicle bandwidth, computing power, channel conditions, and local data volume as input, comprehensively considering the allocation of communication and computing resources. By calculating and ranking the importance scores of each vehicle, it ultimately selects the optimal set of vehicles as output, achieving efficient vehicle selection and model updating under resource-constrained conditions. The specific process includes: ① Information Collection Phase: RSU collects information on all vehicles participating in the Federal Learning program. D n V n , B n h n,k Data, etc. ②Importance parameter calculation stage: The same data collected from each vehicle are normalized, and the importance score is calculated by multiplying the normalization results. ③ Vehicle selection stage: Filter out vehicles whose importance scores do not reach the threshold, and select the remaining vehicles using a greedy algorithm.

4. The vehicle network federated learning system based on structured pruning and knowledge distillation as described in claim 3, characterized in that, Step 4) describes the Grouped Federated Model Aggregation and Recovery algorithm for RSU model recovery aggregation as follows: Based on the federated learning framework, model pruning techniques, and the principle of grouping and aggregating by pruning rate, this approach aims to efficiently aggregate sub-models trained on different vehicles and recover a complete and updated global model to address the heterogeneity of clients in federated networks. The specific process includes: ① Sub-model collection phase: RSU collects all sub-models of the selected vehicles after local training is completed; ② Sub-model recovery phase: RSU uses the pruned indexes of the sub-models to restore the structure of the sub-models to the global model structure; ③ Global Model Aggregation Phase: RSU aggregates the sub-models of the response structure into a new round of global model by using a federated weighted average.

5. The FedPKD system for connected vehicles based on structured pruning and knowledge distillation as described in claim 4 is as follows: Based on the principles of federated learning, personalized knowledge distillation, model pruning, and quantization, this algorithm aims to efficiently train a global model while addressing the computational and communication heterogeneity of vehicle users. The algorithm distributes the global model and pruned sub-models to vehicles, trains them locally using knowledge distillation, and finally reconstructs the aggregated models uploaded by each vehicle.

6. The FedPKD system as described in claim 5, characterized in that, The objective function of the vehicle-to-everything (V2X) federated learning system based on structured pruning and knowledge distillation, as described in step 4), is: The problem of minimizing the energy consumption and model transmission latency for local model training in vehicles can be expressed as: in, 1) This represents the pruning rate of the model, M is the number of vehicles selected to participate in federated learning in each round, η is the weighting coefficient, and T... k It is the total latency of all vehicles participating in federated learning in round k, E k It is the total energy consumption of all vehicles participating in federated learning in round k, T k The calculation formula is as follows: 2) This is the time required for vehicle n to complete parameter training and parameter transmission in the kth round. H is the time required for the k-th round of RSU to receive the model and for model aggregation. n It represents the number of times the local model for vehicle n has been trained. When a vehicle is selected to participate in the k-th round of federated learning, it first receives the corresponding sub-model parameters from the RSU and performs multiple rounds of iterative updates using local data. This represents the time required for the vehicle to train the model locally for each round. After local training is complete, the vehicle uploads the updated model parameters to the RSU. The time required to transmit vehicle model parameters to the RSU; after receiving sub-models uploaded by multiple vehicles, the RSU completes model recovery and global aggregation. This is the time required to receive the sub-model trained on the vehicle side. This is the time required to aggregate the RSU model parameters. The aggregated global model undergoes adaptive structured pruning to improve model lightweighting and communication efficiency. Finally, the pruned global model is redistributed to vehicle nodes. The formula for calculating the above latency is as follows: Among them, D n This represents the amount of local data held by the nth vehicle; This refers to the vehicle's GPU computing power; Let S represent the model compression ratio of vehicle n in the k-th round, and S be the model size. B is the uplink transmission rate of the node; P is the bandwidth. n,k For transmit power and channel gain h n,k Distance d between vehicle and RSU n,k Path loss index The transmission performance of a channel is determined by both the noise power γ and the noise power γ. For edge servers, Here, M represents the server's uplink receiving rate, and M represents the number of vehicles participating in the upload; furthermore, This represents the computational cost required to aggregate the various vehicle models. The server's CPU computing power; 3) Total energy consumption E of all vehicles participating in federated learning in round k k It mainly consists of the following three parts: energy consumption for local model training on the vehicle side. Energy consumption during model parameter upload and energy consumption of RSU receiving vehicle model parameters. And the computational energy consumption of performing global model aggregation at the RSU end. E k The calculation formula is as follows: in, This represents the energy consumption of the m-th vehicle node during the local model training process in the k-th round; This indicates the communication energy consumption of the RSU when receiving model parameters uploaded by each vehicle; Communication energy consumption when uploading model parameters for the m-th vehicle node; α r c is the power factor of the RSU. r This indicates the number of GPU cycles required for each unit of data to be processed by the RSU task. This refers to the processing frequency of the RSU; while S... n The scale of the model aggregation task related to the nth vehicle is represented by the following formula for calculating the energy consumption: Where α n This represents the calculated power factor of vehicle n. The number of GPU cycles required per unit of data for the task. D represents the local computation frequency of vehicle n. n H represents the amount of data that vehicle n needs to process. n P represents the number of local training iterations for vehicle n; n,k Let n be the transmission power of vehicle n during parameter transmission; P represents the parameter transfer time of RSU in task k. n,k This represents the transmission power of the RSU during parameter transmission.

7. The vehicle networking federated learning system based on structured pruning and knowledge distillation as described in any one of claims 1-6, characterized in that, The process of federated training based on this strategy is as follows: 1) Structured Pruning: RSU effectively removes redundant parameters in the model by adopting a structured pruning strategy, thereby reducing the computational overhead required for model training and significantly improving the efficiency of model parameters during transmission. 2) Quantization: During the transmission of model parameters, RSU uses quantization technology to convert high-precision parameters into low-precision representations, thereby significantly reducing the amount of data transmitted and improving communication efficiency. 3) Knowledge Distillation: When training the local model on the vehicle, a knowledge distillation mechanism is introduced to alleviate the performance difference caused by uneven distribution of local data (non-IID), thereby significantly improving the convergence speed and stability of the global model. 4) Model Recovery: During the model aggregation phase, since the sub-model structures uploaded by each vehicle may differ, RSU first restores the sub-models with different structures to a unified global model structure, and then performs the federated aggregation process to ensure the consistency of model parameters and the reliability of aggregation results. 5) Repeat steps 1), 2), 3) and 4) until the accuracy of the global model reaches the target value while minimizing latency and energy consumption.

Images