Priority-based differential privacy client selection policy in vehicle edge computing

Abstract

The invention discloses a client selection and resource allocation strategy for the Internet of Vehicles, which is based on the combination of a Deep Reinforming Learning algorithm and a Federated Learning algorithm. The client selection and resource allocation strategy for the Internet of Vehicles is used for the Internet of Vehicles. The method comprises the following steps: firstly, constructing a three-layer collaborative architecture consisting of a vehicle, an edge server and a cloud server; and secondly, proposing an FLMADDPG algorithm which is based on the combination of Deep Reinformation Learning and Federated Learning and is used for a client selection and resource allocation strategy of the Internet of Vehicles, wherein the FLMADDPG algorithm is used for the client selection of the Internet of Vehicles and the resource allocation strategy of the Internet of Vehicles. And then, obtaining a client selection and resource allocation strategy based on an FLMADDPG algorithm. And finally, when the client is selected to participate in federal aggregation, personalized differential privacy noise is added to the transmitted model parameters.

Description

Technical Field

[0001] This invention relates to the problem of client selection and resource allocation strategies in the context of vehicle-to-everything (V2X) scenarios, specifically to a V2X client selection and resource allocation strategy based on Deep Reinforcement Learning combined with Federated Learning algorithms. Background Technology

[0002] With the evolution of 5G / 6G communication technologies, the Internet of Vehicles (IoV) has become the core of building future intelligent transportation systems. Key applications it supports, such as autonomous driving and high-precision map updates, place extremely high demands on the real-time performance and reliability of data processing. However, the limited computing and storage capabilities of onboard units, coupled with the latency and bandwidth pressure caused by uploading massive amounts of data to remote cloud centers, make traditional cloud computing architectures inadequate.

[0003] To address this, vehicular edge computing (VEC) has emerged. By offloading computing resources to roadside units or base stations, it significantly shortens the data processing path, effectively reduces latency, and alleviates backbone network load, becoming a key technology to meet the stringent QoS requirements of IoV.

[0004] However, while VEC improves efficiency, it also brings new challenges: sensitive data such as vehicle location, trajectory, and behavior, if directly uploaded to a centralized processing center, not only increases the network burden but also makes it more susceptible to privacy leaks due to security vulnerabilities in data aggregation or transmission. In the context of increasingly stringent global data regulations, how to unlock the value of data while protecting user privacy has become a core bottleneck restricting the large-scale deployment of IoV.

[0005] Against this backdrop, in-depth research into how to collaboratively optimize client selection, resource allocation, and privacy protection mechanisms in dynamic and heterogeneous vehicle-to-everything (V2X) environments to achieve a multi-objective balance between low latency, high energy efficiency, and fine-grained privacy and security is particularly urgent and necessary. This is not only crucial for the practical application and performance breakthroughs of federated learning in edge computing scenarios, but also has key practical significance for building a secure, reliable, efficient, and intelligent next-generation intelligent transportation system. Summary of the Invention

[0006] This invention addresses the client selection and resource allocation strategy for applications in the edge network environment of vehicle-to-everything (V2X) networks, providing a client selection and resource allocation strategy based on Deep Reinforcement Learning combined with Federated Learning algorithms.

[0007] This invention is achieved using the following technical solution:

[0008] A strategy for selecting and allocating resources for connected vehicle clients based on Deep Reinforcement Learning combined with Federated Learning algorithms includes the following steps:

[0009] 1) Construct a three-tier collaborative architecture consisting of vehicles, edge servers, and cloud servers;

[0010] 2) Propose the FL_MADDPG algorithm based on Deep Reinforcement Learning combined with Federated Learning for vehicle networking client selection and resource allocation strategies;

[0011] 3) Obtain the client's selection and resource allocation strategy;

[0012] 4) When a client is selected to participate in federated aggregation, add personalized differential privacy noise to the transmitted model parameters.

[0013] In the above technical solution, the collaborative architecture of vehicle, edge server, and cloud server described in step 1) further comprises three parts:

[0014] 1) Vehicle layer: Used to generate computational tasks and train models locally;

[0015] 2) Edge server layer: Provides computing and communication resources for local model aggregation;

[0016] 3) Used to coordinate global model aggregation, verify privacy budgets, and synchronize federated learning processes across multiple edge servers;

[0017] Furthermore, the FL_MADDPG algorithm for client selection and resource allocation strategies described in step 2) is as follows:

[0018] This algorithm is a multi-agent deep reinforcement learning algorithm based on the Actor-Critic architecture. Its core lies in employing a "centralized training, distributed execution" model. During training, all agents can share global state information to optimize the policy, while during decision execution, each agent relies on local observations, thus effectively handling complex interactions and resource competition among multiple agents. We use this algorithm to solve the dynamic client selection problem in a connected vehicle environment.

[0019] During vehicle operation, various tasks arise with differing time and privacy requirements, thus categorizing them into three priority levels. The proposed FL_MADDPG algorithm first models the joint optimization problem as a Markov Decision Process (MDP). Its state space includes the vehicle's task queue and available computing resources between the vehicle and edge servers. The action space jointly determines client selection indicator variables, communication sub-channel allocation variables, and computing resource allocation ratios within the same decision slot. The algorithm aims to optimize task processing time and energy consumption while maximizing task completion rates. Using the state space as input, it generates a suitable resource allocation strategy and a joint strategy (whether to participate in the federated aggregation multi-client selection strategy) as output.

[0020] Furthermore, the objective function of the client selection and resource allocation strategy based on Deep Reinforcement Learning combined with Federated Learning algorithm described in step 3) is:

[0021] ∑ n∈N (F (n,t) ) / (λ n T (n,t) +(1-λ n E (n,t) )+φA t (1)

[0022] in,

[0023] 1) The first part is the overall system cost of the vehicle, which is weighted by time and energy consumption and the number of tasks completed, F. n,t Together they form λ n T (n,t) +(1-λ n E (n,t) λ represents the weighted sum of the vehicle's total time and energy consumption. n This refers to the vehicle's weights over time and energy consumption. The latter part, A... t φ is a scaling factor for the model accuracy in federated learning. The values ​​for latency and energy consumption in the system cost are determined by the client's choice of this strategy.

[0024] 2) If vehicle n is not selected to participate in federated learning, only the local processing time needs to be considered. If vehicle n is selected to participate in federated aggregation, then not only local processing time but also parameter upload and download time need to be considered. And the time spent on aggregation, This indicates the transmission time for sending the model parameters to the edge server. This represents the time spent transmitting model parameters via edge server e to the adjacent edge server e′, and the transmission time for uploading model parameters to the cloud is [missing information]. The specific formula for calculating the above time is as follows:

[0025]

[0026] Among them, V n f represents the computational resources required for the aggregation model parameters. n D represents the computing resources possessed by the vehicle. n Indicates the magnitude of the model parameters. These represent the uplink transmission rates from the vehicle to edge server e, from the vehicle to the adjacent edge server e′, and from edge server e to the cloud server, respectively. This represents the downlink transmission rate from the edge server to the vehicle. When model parameters are transmitted to the server for aggregation, the time spent on aggregation also needs to be considered. These represent the aggregate time spent on edge servers e and e′, respectively. The specific calculation formula is as follows:

[0027]

[0028] Among them, f e with f e′ Let e ​​represent the computing resources possessed by edge server e and its adjacent edge server e′, respectively. and These represent the proportions of computing resources allocated to edge server e and its adjacent edge server e′ for this aggregation, respectively.

[0029] In this process, not only the time spent but also the corresponding energy consumption must be considered. Since energy consumption primarily originates from the transmission process of the wireless communication module, its magnitude is determined by the power of each link and the corresponding transmission time. Consistent with the time model, energy consumption also depends on whether the vehicle is selected and the aggregation path used. Specifically, when vehicle n is not selected to participate in aggregation, only the local computational energy consumption needs to be considered. When vehicle n is selected to participate in the aggregation, the energy consumption incurred during transmission also needs to be considered. and These represent the energy consumption for transmission to the edge server, transmission to adjacent edge servers, and transmission to the cloud, respectively. The specific calculation formula for the above energy consumption is as follows:

[0030]

[0031] Where ζ n This is determined by the CPU architecture of the vehicle's infotainment chip. and These represent the transmission power from the vehicle to the edge server, from the vehicle to the adjacent edge server, and from the edge server to the cloud server, respectively.

[0032] Furthermore, in step 4), when the client is selected to participate in federated aggregation, personalized differential privacy noise is added to the transmitted model parameters as follows:

[0033] 1) In the standard definition of ε-Differential Privacy (ε-DP), if a random mechanism For adjacent datasets that differ by at most one element and And any output subset S, satisfying

[0034]

[0035] This is said to satisfy ε-DP, where ε is a uniform privacy budget. However, in the context of heterogeneous data in connected vehicles, this uniform budget cannot accommodate the differentiated privacy needs of different tasks. Therefore, a personalized differential privacy mechanism is introduced, the core of which is defining a privacy configuration ξ, i.e., a mapping:

[0036] ξ:U→R + (13)

[0037] This mapping maps user i's settings to their personalized privacy preferences ξ. i and with ξ i Instead of a uniform privacy configuration ε, this is defined as ξ-Personalized Differential Privacy (ξ-PDP), which requires a random mechanism. For any records d and d′, the following holds:

[0038]

[0039] To protect privacy, a Laplace1 Mechanism is introduced to inject noise. This mechanism requires computation of the query function. L1 sensitivity It measures the maximum impact of a change to a single data record on the query results, and is defined as:

[0040]

[0041] And by sending the query results Add compliance with this sensitivity Personalized privacy settings i Privacy is achieved using Laplace distributed noise. The added noise is represented as:

[0042]

[0043] 2) Considering that each vehicle has a task queue, the tasks are divided into three categories—high-priority, medium-priority, and low-priority—based on their varying requirements for time and privacy protection. A client selection strategy selects specific clients to participate in federated aggregation. Based on their task queues, each client obtains its own privacy protection level, adds local differential privacy noise to its model parameters, and transmits its privacy protection level encrypted. Upon receiving the model parameters from all selected clients, the server first decrypts the client's local privacy level, obtains the corresponding local privacy budget ξ through a mapping table, and determines whether the current global aggregation result meets the preset global differential privacy threshold ε. max If the condition is not met, global calibration noise is injected again into the aggregated model to make the model meet the differential privacy requirements.

[0044] This invention also provides a process for selecting clients and allocating resources using the above method, and for adding personalized differential privacy noise for privacy protection, as detailed below:

[0045] 1) Federated Learning: The selected client vehicle downloads the current global model W. t The system uses a local dataset for multiple rounds of training. After training, the client configures its privacy settings accordingly. i The personalized differential privacy mechanism described in claim 5 is used to update the model parameters. By adding noise, we obtain the perturbed model parameters. The perturbated model parameters are then compared with the encrypted privacy protection level. The data is uploaded to an edge server for partial aggregation, and then the aggregation result is uploaded to a cloud server. The cloud server performs the final global aggregation, and at this stage checks whether the preset privacy threshold ε is met. max If the condition is not met, global calibration noise is injected, and the final model accuracy A is obtained. t ;

[0046] 2) Deep Reinforcement Learning: By combining vehicle task queue information and the status of each communication link, the reward value given by the environment is derived by considering the time spent, energy consumption, and satisfaction of constraints. The global model accuracy is then added to the reward to obtain a reward value r. t The reward value r t Together with the state of power t Action a to be performed t and new state s t+1 As an empirical tuple (s t ,at ,r t ,s t+1 The data is stored in the experience replay pool. The agent updates its Actor network and Critic network by randomly sampling data from the experience replay pool M, optimizing the joint policy to obtain the maximum cumulative reward in the future;

[0047] 3) Repeat steps 1) and 2) until a client selection and resource allocation strategy that balances the benefits for all parties, with the minimum time and energy consumption, the maximum task completion rate, and the maximum accuracy after client participation in aggregation, is obtained.

[0048] The inventive principle of this invention:

[0049] This invention primarily addresses the client selection and resource allocation problem in the edge network environment of the Internet of Vehicles (IoV). It designs a client selection and resource allocation strategy to reduce system processing latency and energy consumption, improve task completion rate, and simultaneously ensure user privacy and security within a federated learning framework. To this end, Markov decision process modeling, multi-agent deep reinforcement learning, and a personalized differential privacy mechanism based on task priority are introduced to construct a closed-loop decision system that can dynamically adapt to complex environments and collaboratively optimize performance and privacy protection. Based on this, the FL_MADDPG algorithm is proposed. This algorithm innovatively combines the federated learning architecture with multi-agent reinforcement learning and embeds a privacy protection module that dynamically adjusts noise intensity based on task type, achieving joint optimization of client selection, communication path planning, and privacy budget allocation in heterogeneous and dynamic IoV environments.

[0050] The beneficial effects of this invention are as follows:

[0051] This invention proposes a vehicle-to-everything (V2X) client selection strategy based on the collaborative optimization of multi-agent deep reinforcement learning and a task priority-aware privacy protection mechanism. This method effectively helps edge servers make optimal decisions in dynamic and heterogeneous V2X environments, aiming to reduce system processing latency, decrease energy consumption, and improve task completion rates. It introduces Markov decision process modeling and a federated learning architecture to achieve efficient collaborative allocation of communication and computing resources while ensuring user data privacy and security. Furthermore, the proposed FL_MADDPG algorithm innovatively integrates global state information into distributed decision-making and embeds a personalized differential privacy module that dynamically adjusts noise intensity based on task type, ultimately obtaining an intelligent client selection and resource allocation strategy that balances performance and privacy. Attached Figure Description

[0052] Figure 1 It is a three-tier collaborative architecture consisting of vehicles, edge servers, and cloud servers;

[0053] Figure 2 Here is a flowchart example of the FL_MADDPG algorithm;

[0054] Figure 3 A performance comparison of convergence of different algorithms;

[0055] Figure 4 A comparison of the energy consumption performance of different algorithms under different vehicle numbers;

[0056] Figure 5 Performance comparison of different algorithms in terms of task completion rate under different communication and computing resource environments;

[0057] Figure 6 A time performance comparison of different algorithms under different vehicle numbers;

[0058] Figure 7 A comparison of accuracy obtained at different privacy protection levels in the LISA dataset;

[0059] Figure 8 A comparison of accuracy obtained at different privacy protection levels in the MNIST dataset;

[0060] Figure 9 A comparison of accuracy obtained at different privacy protection levels on the CIFAR-10 dataset. Detailed Implementation

[0061] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0062] A strategy for selecting and allocating resources for connected vehicle clients based on Deep Reinforcement Learning combined with Federated Learning algorithms includes the following steps:

[0063] 1) Construct a three-tier collaborative architecture consisting of vehicles, edge servers, and cloud servers;

[0064] 2) Propose the FL_MADDPG algorithm based on Deep Reinforcement Learning combined with Federated Learning for vehicle networking client selection and resource allocation strategies;

[0065] 3) Obtain the client's selection and resource allocation strategy;

[0066] 4) When a client is selected to participate in federated aggregation, add personalized differential privacy noise to the transmitted model parameters.

[0067] The three-layer collaborative architecture proposed in this invention, consisting of vehicles, edge servers, and cloud servers, is as follows: Figure 1 As shown, it mainly includes three parts:

[0068] 1) Vehicle layer: Used to generate computational tasks and train models locally;

[0069] 2) Edge server layer: Provides computing and communication resources for local model aggregation;

[0070] 3) Cloud server layer: used to coordinate global model aggregation, verify privacy budgets, and synchronize federated learning processes across multiple edge servers;

[0071] Figure 2 The FL_MADDPG algorithm flowchart is given. First, random sampling of vehicle terminals is performed, and the model is trained using the local dataset. The accuracy of the global model is aggregated and used as the benefit value for the current vehicle participating in the aggregation. Then, based on the vehicle task queue information and the status of each communication link, the reward value given by the environment is obtained by comprehensively considering the time spent, energy consumption, and the satisfaction of constraints. The obtained global model accuracy is added to the reward as a system cost for the selected client. The policy network is updated through the experience replay pool of the MADDPG algorithm. Through continuous iteration, the optimal client selection and resource allocation strategy is finally obtained.

[0072] The objective function for client selection and resource allocation strategies can be expressed as:

[0073]

[0074] in,

[0075] 1) The first part is the overall system cost of the vehicle, which is weighted by time and energy consumption and the number of tasks completed, F. n,t Together they form λ n T (n,t) +(1-λ n E (n,t) λ represents the weighted sum of the vehicle's total time and energy consumption. n This refers to the vehicle's weights over time and energy consumption. The latter part, A... t φ is a scaling factor for the model accuracy in federated learning. The values ​​for latency and energy consumption in the system cost are determined by the client's choice of this strategy.

[0076] 2) If vehicle n is not selected to participate in federated learning, only the local processing time needs to be considered. If vehicle n is selected to participate in federated aggregation, then not only local processing time but also parameter upload and download time need to be considered. And the time spent on aggregation, This indicates the transmission time for sending the model parameters to the edge server. This represents the time spent transmitting model parameters via edge server e to the adjacent edge server e′, and the transmission time for uploading the model parameters to the cloud is... The specific formula for calculating the above time is as follows:

[0077]

[0078]

[0079] Among them, V n f represents the computational resources required for the aggregation model parameters. n D represents the computing resources possessed by the vehicle. n Indicates the magnitude of the model parameters. These represent the uplink transmission rates from the vehicle to edge server e, from the vehicle to the adjacent edge server e′, and from edge server e to the cloud server, respectively. This represents the downlink transmission rate from the edge server to the vehicle. When model parameters are transmitted to the server for aggregation, the time spent on aggregation also needs to be considered. These represent the aggregate time spent on edge servers e and e′, respectively. The specific calculation formula is as follows:

[0080]

[0081] Among them, f e with f e′ Let e ​​represent the computing resources possessed by edge server e and its adjacent edge server e′, respectively. and These represent the proportions of computing resources allocated to edge server e and its adjacent edge server e′ for this aggregation, respectively.

[0082] In this process, not only the time spent but also the corresponding energy consumption must be considered. Since energy consumption primarily originates from the transmission process of the wireless communication module, its magnitude is determined by the power of each link and the corresponding transmission time. Consistent with the time model, energy consumption also depends on whether the vehicle is selected and the aggregation path used. Specifically, when vehicle n is not selected to participate in aggregation, only the local computational energy consumption needs to be considered. When vehicle n is selected to participate in the aggregation, the energy consumption incurred during transmission also needs to be considered. and These represent the energy consumption for transmission to the edge server, transmission to adjacent edge servers, and transmission to the cloud, respectively. The specific calculation formula for the above energy consumption is as follows:

[0083]

[0084] Where ζn This is determined by the CPU architecture of the vehicle's infotainment chip. and These represent the transmission power from the vehicle to the edge server, from the vehicle to the adjacent edge server, and from the edge server to the cloud server, respectively.

[0085] When a client is selected to participate in federated aggregation, personalized differential privacy noise is added to the transmitted model parameters as follows:

[0086] 1) In the standard definition of ε-Differential Privacy (ε-DP), if a random mechanism For adjacent datasets that differ by at most one element and And any output subset S, satisfying

[0087]

[0088] This is said to satisfy ε-DP, where ε is a uniform privacy budget. However, in the context of heterogeneous data in connected vehicles, this uniform budget cannot accommodate the differentiated privacy needs of different tasks. Therefore, a personalized differential privacy mechanism is introduced, the core of which is defining a privacy configuration ξ, i.e., a mapping:

[0089] ξ:U→R + (13)

[0090] This mapping maps user i's settings to their personalized privacy preferences ξ. i and with ξ i Instead of a uniform privacy configuration ε, this is defined as ξ-Personalized Differential Privacy (ξ-PDP), which requires a random mechanism. For any records d and d′, the following holds:

[0091]

[0092] To protect privacy, a Laplace1 Mechanism is introduced to inject noise. This mechanism requires computation of the query function. L1 sensitivity It measures the maximum impact of a change to a single data record on the query results, and is defined as:

[0093]

[0094] And by sending the query results Add compliance with this sensitivity Personalized privacy settings iPrivacy is achieved using Laplace distributed noise. The added noise is represented as:

[0095]

[0096] 2) Considering that each vehicle has a task queue, the tasks are divided into three categories—high-priority, medium-priority, and low-priority—based on their varying requirements for time and privacy protection. A client selection strategy selects specific clients to participate in federated aggregation. Based on their task queues, each client obtains its own privacy protection level, adds local differential privacy noise to its model parameters, and transmits its privacy protection level encrypted. Upon receiving the model parameters from all selected clients, the server first decrypts the client's local privacy level, obtains the corresponding local privacy budget ξ through a mapping table, and determines whether the current global aggregation result meets the preset global differential privacy threshold ε. max If the condition is not met, global calibration noise is injected again into the aggregated model to make the model meet the differential privacy requirements.

[0097] This invention also provides a process for selecting clients and allocating resources using the above method, and for adding personalized differential privacy noise for privacy protection, as detailed below:

[0098] 1) Federated Learning: The selected client vehicle downloads the current global model W. t The system uses a local dataset for multiple rounds of training. After training, the client configures its privacy settings accordingly. i The personalized differential privacy mechanism described in claim 5 is used to update the model parameters. By adding noise, we obtain the perturbed model parameters. The perturbated model parameters are then compared with the encrypted privacy protection level. The data is uploaded to an edge server for partial aggregation, and then the aggregation result is uploaded to a cloud server. The cloud server performs the final global aggregation, and at this stage checks whether the preset privacy threshold ε is met. max If the condition is not met, global calibration noise is injected, and the final model accuracy A is obtained. t ;

[0099] 2) Deep Reinforcement Learning: By combining vehicle task queue information and the status of each communication link, the reward value given by the environment is derived by considering the time spent, energy consumption, and satisfaction of constraints. The global model accuracy is then added to the reward to obtain a reward value r. t The reward value r t Together with the state of power t Action a to be performedt and new state s t+1 As an empirical tuple (s t ,a t ,r t ,s t+1 The data is stored in the experience replay pool. The agent updates its Actor network and Critic network by randomly sampling data from the experience replay pool M, optimizing the joint policy to obtain the maximum cumulative reward in the future;

[0100] 3) Repeat steps 1) and 2) until a client selection and resource allocation strategy that balances the benefits for all parties, with the minimum time and energy consumption, the maximum task completion rate, and the maximum accuracy after client participation in aggregation, is obtained.

[0101] Simulation results

[0102] The method of this invention is compared with three different deep reinforcement learning algorithms: DDPG, PPO, and DQN. Specifically, the comparison criteria are convergence speed, energy consumption, task completion rate, latency, and the degree of impact on the model under different levels of noise.

[0103] Figure 3 The comparison shows that although the FL MADDPG algorithm proposed in this invention has a slower convergence speed than DQN, its final average reward value is significantly higher than that of PPO, DQN and DDPG, proving that it can make better global decisions in long-term operation.

[0104] Figure 4 The comparison shows that the algorithm proposed in this invention exhibits the lowest normalized energy consumption under different device scales. Its advantage stems from its ability to perceive the global system state, achieving more refined resource allocation and higher energy efficiency.

[0105] Figure 5 The comparison shows that the FL MADDPG algorithm maintains the highest task completion rate under all combinations of communication and computing resources, and its advantage is most obvious when bandwidth is sufficient, which verifies its high efficiency in resource utilization.

[0106] Figure 6 The comparison shows that as the number of devices increases, the latency growth of the FL MADDPG algorithm is the most gradual and consistently the lowest, and it also achieves the best task completion rate in most scenarios. Its slightly lower completion rate under high concurrency is due to offloading some tasks to the cloud to ensure overall stability, reflecting its trade-off between performance and stability.

[0107] Figure 7 , 8As can be seen from the comparison in Figures 9 and 1, the task priority-aware personalized differential privacy mechanism proposed in this invention is effective. The model accuracy in the hybrid privacy scenario (SEC 3) is between high privacy and no privacy, and the convergence process is more stable, proving that the dynamic balancing strategy of compensating for noise in the high privacy client by high-quality updates from the low privacy client can meet diverse privacy needs without sacrificing global performance.

Claims

1. A strategy for selecting and allocating resources for connected vehicle clients based on Deep Reinforcement Learning combined with Federated Learning algorithms, characterized in that, Includes the following steps: 1) Construct a three-tier collaborative architecture consisting of vehicles, edge servers, and cloud servers; 2) Propose the FL_MADDPG algorithm based on Deep Reinforcement Learning combined with Federated Learning for vehicle networking client selection and resource allocation strategies; 3) Obtain the client's selection and resource allocation strategy; 4) When a client is selected to participate in federated aggregation, add personalized differential privacy noise to the transmitted model parameters.

2. The vehicle-to-everything (V2X) client selection and resource allocation strategy based on Deep Reinforcement Learning combined with Federated Learning algorithm as described in claim 1, characterized in that, The heterogeneous computing architecture consisting of vehicles, edge servers, and cloud servers described in step 1) comprises three parts: 1) Vehicle layer: Used to generate computational tasks and train models locally; 2) Edge server layer: Provides computing and communication resources for local model aggregation; 3) Cloud server layer: used to coordinate global model aggregation, verify privacy budgets, and synchronize federated learning processes among multiple edge servers.

3. The vehicle-to-everything (V2X) client selection and resource allocation strategy based on Deep Reinforcement Learning combined with Federated Learning algorithm as described in claim 2, characterized in that, The FL_MADDPG algorithm for selecting vehicle network clients and allocating resources, as described in step 2), is as follows: This algorithm is a multi-agent deep reinforcement learning algorithm based on the Actor-Critic architecture. Its core lies in adopting a "centralized training and distributed execution" mode. During training, all agents can share global state information to optimize the policy, while each agent relies on local observations when making decisions. This can effectively handle the complex interaction and resource competition problems between multiple agents. We use this algorithm to solve the dynamic client selection problem in the Internet of Vehicles environment. During vehicle operation, various tasks arise with varying time and privacy requirements. These tasks are categorized into three priority levels. The proposed FL_MADDPG algorithm first models the joint optimization problem as a Markov Decision Process (MDP). Its state space includes the vehicle's task queue and the available computing resources between the vehicle and the edge server. The action space jointly determines the client selection indicator variable, communication sub-channel allocation variable, and computing resource allocation ratio within the same decision slot. The goal is to optimize task processing time and energy consumption while maximizing task completion rate. Using the state space as input, the algorithm generates a suitable resource allocation strategy and a joint strategy for whether to participate in the federated aggregation multi-client selection strategy as output.

4. The vehicle-to-everything (V2X) client selection and resource allocation strategy as described in claim 3, characterized in that, The objective function for the client selection and resource allocation strategy based on Deep Reinforcement Learning combined with Federated Learning algorithm, as described in step 3), is: ∑ n∈N (F (n,t) ) / (λ n T (n,t) +(1-l n )E (n,t) )+φA t (1) in, 1) The first part is the overall system cost of the vehicle, which is weighted by time and energy consumption and the number of tasks completed, F. n,t Together they form λ n T (n,t) +(1-λ n E (n,t) λ represents the weighted sum of the vehicle's total time and energy consumption. n It is the vehicle's weighting over time and energy consumption; the latter part A t For the model accuracy of federated learning, φ is a scaling factor; in the system cost, the values ​​of latency and energy consumption are determined by the client's choice of this strategy; 2) If vehicle n is not selected to participate in federated learning, only the local processing time needs to be considered. If vehicle n is selected to participate in federated aggregation, then not only local processing time but also parameter upload and download time need to be considered. And the time spent on aggregation, This indicates the transmission time for sending the model parameters to the edge server. This represents the time spent transmitting model parameters via edge server e to the adjacent edge server e′, and the transmission time for uploading the model parameters to the cloud is... The specific formula for calculating the above time is as follows: Among them, V n f represents the computational resources required for the aggregation model parameters. n D represents the computing resources possessed by the vehicle. n Indicates the magnitude of the model parameters. These represent the uplink transmission rates from the vehicle to edge server e, from the vehicle to the adjacent edge server e′, and from edge server e to the cloud server, respectively. This represents the downlink transmission rate from the edge server to the vehicle; when model parameters are transmitted to the server for aggregation, the time spent on aggregation also needs to be considered. These represent the aggregate time spent on edge servers e and e′, respectively, and the specific calculation formula is as follows: Among them, f e with f e′ Let e ​​represent the computing resources possessed by edge server e and its adjacent edge server e′, respectively. These represent the proportions of computing resources allocated to edge server e and its adjacent edge server e′ for this aggregation, respectively. In this process, not only the time spent but also the corresponding energy consumption must be considered. Since energy consumption mainly originates from the transmission process of the wireless communication module, its magnitude is determined by the power of each link and the corresponding transmission time. Consistent with the time model, energy consumption also depends on whether the vehicle is selected and the aggregation path used. Specifically, when vehicle n is not selected to participate in aggregation, only the local computational energy consumption needs to be considered. When vehicle n is selected to participate in the aggregation, the energy consumption incurred during transmission also needs to be considered. and These represent the energy consumption for transmission to the edge server, transmission to adjacent edge servers, and transmission to the cloud, respectively. The specific calculation formulas for the above energy consumption are as follows: Where ζ n This is determined by the CPU architecture of the vehicle's infotainment chip. and These represent the transmission power from the vehicle to the edge server, from the vehicle to the adjacent edge server, and from the edge server to the cloud server, respectively.

5. The vehicle-to-everything (V2X) client selection and resource allocation strategy based on Deep Reinforcement Learning combined with Federated Learning algorithm as described in claim 4, characterized in that, Step 4) When the client is selected to participate in federated aggregation, add personalized differential privacy noise to the transmitted model parameters as follows: 1) In the standard definition of ε-Differential Privacy (ε-DP), if a random mechanism For adjacent datasets that differ by at most one element and And any output subset S, satisfying This is said to satisfy ε-DP, where ε is a uniform privacy budget; however, in the scenario of heterogeneous data in the Internet of Vehicles, this uniform budget cannot take into account the differentiated privacy needs of different tasks; therefore, a personalized differential privacy mechanism is introduced, the core of which is to define a privacy configuration ξ, i.e., a mapping: ξ:U→R + (13) This mapping maps user i's settings to their personalized privacy preferences ξ. i and with ξ i Instead of a uniform privacy configuration ε, this definition is ξ-Personalized Differential Privacy (ξ-PDP), which requires a random mechanism. For any records d and d′, the following holds: To protect privacy, Laplace1 Mechanism is introduced to inject noise; This mechanism requires calculating the query function. L1 sensitivity It measures the maximum impact of a change to a single data record on the query results, and is defined as: And by sending the query results Add compliance with this sensitivity Personalized privacy settings i Privacy is achieved using Laplace distributed noise, and the added noise is represented as follows: 2) Considering that each vehicle has a task queue, the tasks are divided into three categories—high-priority, medium-priority, and low-priority—based on their varying requirements for time and privacy protection. A client selection strategy is used to select specific clients to participate in federated aggregation. Based on their task queues, each client obtains its own privacy protection level, adds local differential privacy noise to its model parameters, and transmits its privacy protection level encrypted. Upon receiving the model parameters from all selected clients, the server first decrypts the client's local privacy level, obtains the corresponding local privacy budget ξ through a mapping table, and determines whether the current global aggregation result meets the preset global differential privacy threshold ε. max If the condition is not met, global calibration noise is injected again into the aggregated model to make the model meet the differential privacy requirements.

6. The vehicle-to-everything (V2X) client selection and resource allocation strategy based on Deep Reinforcement Learning combined with Federated Learning algorithm as described in any one of claims 1-5, characterized in that, The process of selecting clients based on this strategy is as follows: 1) Federated Learning: The selected client vehicle downloads the current global model W. t The system uses a local dataset for multiple rounds of training. After training, the client configures its privacy settings accordingly. i The personalized differential privacy mechanism described in claim 5 is used to update the model parameters. By adding noise, we obtain the perturbed model parameters. The perturbated model parameters are then compared with the encrypted privacy protection level. The data is uploaded to an edge server for local aggregation, and then the aggregation result is uploaded to a cloud server. The cloud server performs the final global aggregation and checks whether the preset privacy threshold ε is met at this stage. max If the condition is not met, global calibration noise is injected, and the final model accuracy A is obtained. t ; 2) Deep Reinforcement Learning: By combining vehicle task queue information and the status of each communication link, the reward value given by the environment is derived by considering the time spent, energy consumption, and satisfaction of constraints. The global model accuracy is then added to the reward to obtain a reward value r. t The reward value r t Together with the state of power t Action a to be performed t and new state s t+1 As an empirical tuple (s t ,a t ,r t ,s t+1 Stored in the experience replay pool; The agent updates its Actor and Critic networks by randomly sampling data from the experience replay pool M, optimizing the joint policy to obtain the maximum cumulative reward in the future. 3) Repeat steps 1) and 2) until a client selection and resource allocation strategy that balances the benefits for all parties, with the minimum time and energy consumption, the maximum task completion rate, and the maximum accuracy after client participation in aggregation, is obtained.

Images