A Resource Allocation Method for Vehicle Networks Based on Federated Learning

By constructing a hierarchical federated learning system model and the SAC reinforcement learning framework, the vehicle selection and transmission power allocation are optimized, solving the problems of dynamic channel changes and data heterogeneity in vehicle-to-everything (V2X) federated learning, and improving the model convergence speed and resource utilization.

CN123123847APending Publication Date: 2026-11-17CHONGQING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611401663.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-10
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing federated learning methods for vehicle networks, which employ random vehicle polling and fixed transmit power allocation, are ill-suited to the constraints of highly dynamic channel changes, heterogeneous data distribution, and limited vehicle dwell time. This results in slow model convergence, frequent communication interruptions, and wasted system resources.

Method used

A hierarchical federated learning system model is constructed, which is combined with the soft actor-critic (SAC) reinforcement learning decision framework. A multi-dimensional comprehensive scoring mechanism is used to select vehicles and dynamically adjust the transmission power to optimize vehicle selection and transmission power allocation. A composite state space and exploratory annealing smoothing technique are designed to adapt to the dynamic changes of the system.

Benefits of technology

It significantly improves the convergence speed and generalization performance of the global model, reduces the total system energy consumption and training latency, improves resource utilization, and provides robust resource allocation support.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123847A_ABST
    Figure CN123123847A_ABST
Patent Text Reader

Abstract

This invention belongs to the field of mobile communication and vehicle-to-everything (V2X) technology, specifically relating to a resource allocation method for V2X based on federated learning. This method constructs an edge-cloud hierarchical federated learning system model, including evaluation models for communication, latency, energy consumption, and data heterogeneity, and configures dual disconnection judgment rules. It constructs a joint vehicle selection and transmit power allocation optimization problem with the objective of minimizing the weighted total cost, transforming it into a Markov decision process. A SAC reinforcement learning agent is employed, which dynamically perceives changes in the composite state space system and outputs multi-dimensional actions using exploratory annealing and action smoothing mechanisms. These actions are used to calculate the comprehensive score of candidate vehicles to filter high-value vehicles and to dynamically adjust the transmit power of selected vehicles. Federated training, disconnection filtering, edge and global aggregation are iteratively executed, and the agent network parameters are updated based on the reward function. This invention effectively improves the model convergence speed and training stability, and significantly reduces system energy consumption and communication overhead.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of mobile communication and vehicle networking technology, specifically relating to a vehicle networking resource allocation method based on federated learning. Background Technology

[0002] With the rapid development of vehicle-to-everything (V2X) and autonomous driving technologies, vehicle terminals have generated massive amounts of perception data, providing a data foundation for applications such as path planning, collaborative obstacle avoidance, and traffic flow prediction in intelligent transportation systems. However, this data is scattered across various vehicle nodes and involves sensitive information such as location trajectories and driving behavior, making it difficult to directly aggregate it to the cloud for centralized model training. Federated learning, as a distributed machine learning paradigm, allows vehicles to train models locally using their own data, uploading only model parameters rather than raw data. This protects user privacy while enabling collaborative training of global models, and is therefore widely regarded as a key technological path for achieving swarm intelligence in V2X scenarios.

[0003] Currently, the mainstream approach to applying federated learning to connected vehicles (V2V) involves roadside units or base stations acting as edge servers, distributing global models to vehicles within their coverage area. Each vehicle performs several rounds of stochastic gradient descent training using local data, then uploads its model parameters to the edge server. The edge server aggregates the regional models and further uploads them to the central server for global updates. Within this framework, vehicle selection typically employs random selection or a round-robin mechanism, and participating vehicles use fixed transmission power and preset computing frequencies for local training and communication uploads. While this approach has proven effective in implementing basic federated learning functionalities in practical deployments, its design primarily follows traditional federated averaging algorithms, focusing on balanced data sampling and lacking specific consideration for the time-varying nature of the V2V wireless environment and vehicle mobility.

[0004] However, the existing solutions still have the following shortcomings in real-world, highly dynamic vehicle-to-everything (V2X) scenarios: Firstly, the high-speed movement of vehicles causes rapid changes in channel conditions, making it difficult for fixed transmission power and preset computing frequencies to adapt to dynamic communication conditions, potentially leading to transmission failures or resource waste. Secondly, the local data of different vehicles exhibits non-independent and identically distributed characteristics, and randomly selecting vehicles for training introduces a large amount of low-quality or redundant data, severely slowing down the global model convergence speed. Furthermore, the dwell time of vehicles within the coverage area of ​​edge servers is limited. If the total latency of local training and model upload exceeds the remaining dwell time, the training results for that vehicle become invalid due to the inability to upload, resulting in a waste of computing and communication resources. Therefore, how to jointly optimize vehicle selection and transmission power allocation under the constraints of channel time-varying characteristics, data heterogeneity, and dwell time to improve the convergence efficiency and resource utilization of federated learning is an urgent problem to be solved. Summary of the Invention

[0005] The technical problem to be solved by this invention is that existing federated learning methods for vehicle networks, due to their use of random vehicle polling and fixed transmit power allocation, are difficult to adapt to the constraints of highly dynamic channel changes, heterogeneous data distribution, and limited vehicle dwell time, resulting in slow model convergence, frequent communication interruptions, and waste of system resources.

[0006] To address the aforementioned technical issues, this invention provides a resource allocation method for vehicle-to-everything (V2X) networks based on federated learning. This method is applied to a layered V2X federated learning architecture consisting of vehicles, edge servers, and a central server. The method includes a pre-training preparation phase and a cyclic iterative training phase.

[0007] The following steps should be performed during the training preparation phase:

[0008] S1: Construct a hierarchical federated learning system model; the constructed hierarchical federated learning system model includes a communication model, a latency model, an energy consumption model, and a data heterogeneity evaluation model; calculate the data quality score of each vehicle through the data heterogeneity evaluation model, and configure a dual disconnection judgment rule, the local model output by the vehicle that is judged to be disconnected will not participate in the model aggregation in this round;

[0009] S2: Construct an optimization problem for joint vehicle selection and transmit power allocation; the optimization objective of the optimization problem is to minimize the weighted total cost, which consists of the total latency of a single round of federated learning, the total energy consumption of the system, and the data utility of successfully uploaded samples; the optimization variables of the optimization problem include a subset of vehicles to be selected and the transmit power vector of the corresponding selected vehicles;

[0010] S3: The optimization problem is transformed into a Markov decision process, and the SAC agent is initialized. The state space and continuous action space are defined. The action output of the SAC agent is processed using exploratory annealing and action smoothing mechanisms to obtain the processed action output. The comprehensive score of the candidate vehicles is calculated based on the processed action output. The vehicles participating in the training round are selected based on the comprehensive score. The transmission power of the selected vehicles is obtained by mapping the processed action output. A reward function is constructed. The reward function calculates the reward in real time by comprehensively considering the total latency of a single round of federated learning, the disconnection rate of the selected training vehicles, the total system energy consumption, and the data utility of successfully uploaded samples.

[0011] In the iterative training phase, each round utilizes the hierarchical federated learning system model obtained in the pre-training preparation phase, the dual disconnection judgment rule, and the initialized SAC agent to perform the following steps:

[0012] S4: The central server distributes the global model to each edge server, and the edge server distributes federated learning requests to the vehicles within its coverage area; the vehicles that receive the requests and respond report their own status information according to the hierarchical federated learning system model, and the edge server generates a candidate vehicle set.

[0013] S5: Call the SAC agent to output actions based on the candidate vehicle set, obtain the selected vehicle set and the corresponding vehicle's transmission power; the selected vehicles perform local training using local data; filter vehicles for disconnection according to the dual disconnection judgment rule; the edge server performs weighted aggregation on the valid local models uploaded by the non-disconnected vehicles to obtain the edge model, and sends the edge model back to the central server; the central server receives the edge models sent back by each edge server, performs federated average aggregation, and obtains the updated global model;

[0014] S6: Store the state transition tuple generated in this round of interaction into the experience replay buffer, sample a batch of samples from the experience replay buffer, and update the parameters of the Actor network, dual Critic network, and target network of the SAC agent.

[0015] S7: Determine whether the current communication round has reached the preset global training round; if the preset global training round has not been reached, return to step S4 to continue iterative execution; if the preset global training round has been reached, end the training and output the converged global model.

[0016] The present invention has at least the following beneficial effects

[0017] This invention achieves joint dynamic optimization of vehicle selection and transmit power by constructing a reinforcement learning decision framework based on soft actor-commentator (SAC). On the one hand, a multi-dimensional comprehensive scoring mechanism integrating information age, dwell time, channel gain, and data quality score is designed, enabling the system to proactively identify and prioritize vehicles with high data value and low disconnection risk for aggregation, effectively suppressing the negative impact of non-independent and identically distributed data on convergence accuracy. On the other hand, the continuous adjustment factor output by the SAC agent adaptively controls the uplink transmit power of vehicles, and combines dual disconnection judgment rules to implement real-time filtering of vehicles that have timed out or experienced channel degradation, significantly reducing communication interruptions and computational resource waste in mobile scenarios. Simultaneously, the composite state space and exploratory annealing smoothing techniques employed endow the agent with the ability to perceive the dynamic changes in the system, ensuring sufficient exploration of the strategy in the early stages of training and stable execution in the later stages. Ultimately, while reducing the total system energy consumption and training latency, the convergence speed and generalization performance of the global model are significantly improved, providing robust and efficient resource allocation support for highly dynamic vehicular network federated learning tasks. Attached Figure Description

[0018] Figure 1This is a schematic diagram of the method flow of the present invention;

[0019] Figure 2 A comparative graph showing how the model loss value of different algorithms changes with the number of communication rounds during the training process;

[0020] Figure 3 A comparative graph showing how data quality changes with the number of communication rounds during the training process for different algorithms;

[0021] Figure 4 A comparative graph showing how the disconnection rate of different algorithms changes with the number of communication rounds during training;

[0022] Figure 5 A comparative graph showing how real-time rewards change with the number of communication rounds during the training process for different algorithms;

[0023] Figure 6 This is a comparison curve showing the accuracy of different algorithms under cumulative latency and energy consumption constraints. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] like Figure 1 As shown, the present invention provides a resource allocation method for vehicle-to-everything (V2X) networks based on federated learning, which is applied to a layered V2X federated learning architecture consisting of vehicles, edge servers, and central servers. The method includes a pre-training preparation stage and a cyclic iterative training stage.

[0026] In this embodiment, the federated learning architecture comprises M edge servers, deployed on roadside units, each with a coverage radius of M. During the simulation, vehicles travel along the road and sequentially pass through the coverage areas of each edge server. At the start of each round of federated learning, the vehicles located within the coverage area of ​​each edge server and that respond to the federated learning request constitute the candidate vehicle set for that round. Candidate vehicle set The candidate vehicle set changes dynamically wheel by wheel as the vehicle's position changes. The number of candidate vehicles is denoted by N, which changes with each round of federated learning. Vehicles leave the edge server's coverage area after completing the corresponding training round and do not participate in subsequent rounds. The local dataset for each vehicle is generated from the CIFAR-10 image classification dataset, partitioned according to a Dirichlet distribution to simulate the non-independent and identically distributed nature of data in a connected vehicle environment. In this embodiment, the local and global models are convolutional neural networks used for image classification; the training samples are image data, and the sample labels are the categories to which the images belong.

[0027] The following steps should be performed during the training preparation phase:

[0028] S1: Construct a hierarchical federated learning system model; the constructed hierarchical federated learning system model includes a communication model, a latency model, an energy consumption model, and a data heterogeneity evaluation model; calculate the data quality score of each vehicle through the data heterogeneity evaluation model, and configure a dual disconnection judgment rule, the local model output by the vehicle that is judged to be disconnected will not participate in the model aggregation in this round;

[0029] Preferably, the communication model adopts the orthogonal frequency division multiple access (OFDMA) communication method to calculate the channel gain between the vehicle and the edge server, the uplink signal-to-interference-plus-noise ratio (SINORR) of the vehicle uploading the local model to the edge server, and the corresponding uplink transmission rate.

[0030] The latency model is used to calculate the following latency: vehicle local model training latency, vehicle receiving latency of the global model sent by the edge server, vehicle uploading latency of the trained local model to the edge server, edge aggregation latency of the edge server aggregating the local models uploaded by each vehicle, edge server backhaul latency of the aggregated edge model to the central server, and central server aggregating the edge models uploaded by each edge server. The total vehicle training latency is the sum of the vehicle local model training latency, the vehicle receiving latency of the global model sent by the edge server, and the vehicle uploading latency of the trained local model to the edge server.

[0031] The energy consumption model is used to calculate the computational energy consumption generated by the vehicle performing local model training, the transmission communication energy consumption generated by the vehicle uploading the local model to the edge server, the receiving energy consumption generated by the vehicle receiving the global model sent by the edge server, the edge aggregation computational energy consumption generated by the edge server performing weighted aggregation operation on the local model uploaded by the vehicle, and the return transmission energy consumption generated by the edge server returning the edge model to the central server.

[0032] The dual disconnection determination rule includes: when the total training latency of the vehicle is greater than the predicted remaining dwell time of the vehicle within the coverage area of ​​its home edge server, or when the uplink signal-to-interference-plus-noise ratio between the vehicle and its home edge server is lower than a preset signal-to-interference-plus-noise ratio threshold, the vehicle is determined to be disconnected; the local model output by the vehicle determined to be disconnected does not participate in the current round of model aggregation.

[0033] Preferably, the data heterogeneity assessment model uses KL divergence to calculate the vehicle's data quality score, including:

[0034] vehicle After responding to the federated learning request issued by the edge server, the edge server will distribute the global category distribution issued by the central server to the vehicles. ;vehicle Calculate vehicle categories based on global category distribution and category distribution of local data. Data quality score :

[0035]

[0036]

[0037] in, Represents an exponential function; Indicates vehicle Category Divergence; This indicates the total number of categories in the data sample; Indicates vehicle Categories in local data samples The proportion; Represents the categories in the global data sample The percentage.

[0038] In this embodiment, the central server is deployed in the cloud and connected to all edge servers via a backhaul link. The central server performs global model aggregation, which involves collecting edge models uploaded by each edge server, weighting and aggregating them using a federated averaging algorithm to generate an updated global model, and then distributing the global model to each edge server to initiate the next round of federated learning. In this embodiment, the CPU computing frequency of the central server is [missing information]. The computational complexity coefficient of the global aggregation algorithm is .

[0039] In this embodiment, M edge servers are deployed in a roadside unit (RSU), and each edge server... The position coordinates are The coverage radius is The antenna height of the edge server is Each edge server uses Orthogonal Frequency Division Multiple Access (OFDMA) communication technology to communicate with vehicles within its coverage area. Each edge server has... There are [number] resource blocks, and the bandwidth of each resource block is [bandwidth]. Downlink bandwidth is The backhaul link bandwidth is The edge server's CPU frequency is... The aggregation computation complexity coefficient is Downlink transmit power is The return transmission power is The backhaul channel gain from the edge server to the central server is... .

[0040] In this embodiment, the vehicles are mobile nodes that travel along roads and sequentially pass through the coverage areas of each edge server. Each vehicle is equipped with a Global Positioning System (GPS) receiver, an Inertial Measurement Unit (IMU), and an onboard communication unit, enabling it to acquire its own position coordinates, speed, acceleration, and other information in real time. At the start of each round of federated learning, vehicles within the coverage area of ​​the edge servers, upon receiving a federated learning request, autonomously decide whether to respond based on their own status (such as currently available computing resources and data volume). Vehicles that choose to respond acquire their current status information through onboard sensors and report it to their home edge server.

[0041] vehicle At the present moment Location information is obtained through an in-vehicle GPS / IMU integrated navigation system, and its three-dimensional coordinates are as follows: , Indicates vehicle The vertical coordinates, which include the road height and the vehicle mounting height; and Indicates vehicle Position coordinates; velocity and acceleration Acquired by vehicle speed sensors and IMU. The vehicle's local dataset size is [size missing]. The local data category distribution is as follows vehicle When responding to a federated learning request, it reports its own status information, including its current position coordinates, velocity, acceleration, and available CPU computing frequency. Local dataset size Local data category distribution and locally pre-calculated data quality scores.

[0042] The number of vehicle nodes within the coverage area of ​​each edge server can be defined as ;vehicle With edge servers Euclidean distance between At that time, the vehicle As an edge server If there are vehicles at the service node. With multiple edge servers Euclidean distance between Then the vehicle As Euclidean distance Minimal edge server The service nodes, thus forming distinct vehicle candidate sets under each edge server; vehicles With edge servers Euclidean distance between ;

[0043] At the start of each round of federated learning, the central server sends the current global model and federated learning request to each edge server. Upon receiving this request, the edge server forwards it to all vehicles within its coverage area. Vehicles within its coverage area autonomously decide whether to respond to the request—vehicles that choose to respond report their own status information to their home edge server. The edge server only adds vehicles that have chosen to respond and completed their status information upload to the candidate vehicle set for this round. Vehicles that do not respond or whose status information upload is incomplete will not participate in this round of vehicle selection and local training.

[0044] vehicle With edge servers Channel gain between Calculated from path loss:

[0045]

[0046] in, Indicates vehicle With edge servers The equivalent path loss between them; the wireless link between the vehicle and the edge server includes line-of-sight (LOS) transmission states and non-line-of-sight (NLOS) transmission states; when the vehicle With edge servers When in line-of-sight transmission mode, Line-of-sight transmission path loss When the two are in a non-line-of-sight transmission state, Non-line-of-sight transmission path loss .therefore, and represents the conditional path loss under the two propagation states, while PL represents the actual path loss determined based on the current propagation state.

[0047] vehicle With edge servers The probability that they are in a line-of-sight transmission state is:

[0048]

[0049] In line-of-sight transmission mode, the path loss between vehicle n and edge server m is:

[0050]

[0051] In non-line-of-sight transmission mode, the path loss between vehicle n and edge server m is:

[0052]

[0053] in, This represents the line-of-sight transmission probability; the non-line-of-sight transmission probability is the complement of the line-of-sight transmission probability, that is, the sum of the two is 1. and These represent the path loss in line-of-sight transmission mode and non-line-of-sight transmission mode, respectively. Indicates vehicle With edge servers The distance between them; This indicates the 5.9GHz dedicated frequency band for vehicle-to-everything (V2X) communication. Shadow fading in scenes with limited view distance; Shadow fading in non-view-of-sight scenes;

[0054] Let the first Edge servers in federated learning The selected vehicle set is ,gather The number of vehicles in China is ;when At this time, each vehicle can be allocated an independent resource block, and there is no co-channel interference; when At that time, some vehicles will reuse the same resource block, thus causing co-channel interference; Wheels assigned to vehicles The resource block number is denoted as .vehicle In the Wheel-to-Home Edge Server Uplink signal-to-noise ratio of the uploaded local model Calculate using the following formula:

[0055]

[0056] in, Indicates vehicle No. The wheel's transmission power; Indicates vehicle To the edge server Channel gain, For additive white Gaussian noise power, Indicates the first Wheels assigned to vehicles The resource block number; Indicates the first Wheel edge server The selected set of vehicles; Indicates the first Wheels assigned to vehicles The resource block number;

[0057] According to Shannon's formula, vehicles To edge servers The uplink transmission rate for uploading local models is:

[0058]

[0059] Edge server To the vehicle Downlink data transmission rate of the global model for:

[0060]

[0061] in, This indicates the downlink bandwidth of the edge server; Represents edge server downlink transmit power;

[0062] Edge server The rate at which the edge model is uploaded to the central server for:

[0063]

[0064] in, Represents edge server Upload the backhaul link bandwidth of the edge model to the central server; Represents edge server The return transmission power; The channel gain from the edge server to the central server;

[0065] The delay model is used to calculate the following delay components respectively:

[0066] vehicle To edge servers Model upload latency for uploaded local models that have completed training for:

[0067]

[0068] in, This represents the total number of bits in the local model parameters;

[0069] vehicle Model reception latency of receiving the global model distributed by the edge server for:

[0070]

[0071] vehicle Local model training latency is :

[0072]

[0073] in, Indicates vehicle The number of CPU cycles required to process each sample; Indicates vehicle In the The CPU computing frequency used for local training; The local training rounds can be set to 2 in this embodiment;

[0074] Edge server Model backhaul latency of sending the aggregated edge model back to the central server for:

[0075]

[0076] Edge server Edge aggregation latency of uploading local models from various vehicles for:

[0077]

[0078] in, For edge servers The number of vehicles that successfully completed local training within the coverage area; This represents the computational complexity coefficient of the edge aggregation algorithm. This refers to the CPU frequency of the edge server.

[0079] The central server aggregates the global aggregate latency of the edge models uploaded by each edge server. for:

[0080]

[0081] in, The number of edge servers required to complete the edge model upload. This represents the computational complexity coefficient of the global aggregation algorithm. This represents the computational complexity coefficient of the global aggregation algorithm. The CPU frequency of the central server;

[0082] vehicle Total training latency for:

[0083]

[0084] The total latency for a single round of federated learning is:

[0085]

[0086] in, Indicates the first Total latency of round-fed learning; This represents the set of all vehicles that have successfully uploaded their local models, i.e., the set of all vehicles on all edge servers that have successfully uploaded their local models. This indicates that the edge server set of the edge model has been successfully uploaded;

[0087] The energy consumption model is used to calculate the following energy consumption components separately:

[0088] vehicle Computational energy consumption during local model training for:

[0089]

[0090] in, This represents the effective capacitance coefficient of the vehicle's CPU.

[0091] vehicle Transmission communication energy consumption generated when vehicles upload local models to edge servers for:

[0092]

[0093] vehicle Receive energy consumption generated by the global model distributed by the edge server for:

[0094]

[0095] in, The receiving power of the vehicle;

[0096] vehicle Total energy consumption for:

[0097]

[0098] Edge server Energy consumption of edge aggregation computing generated by performing weighted aggregation operations on local vehicle models for:

[0099]

[0100] in, The effective capacitance coefficient of the CPU in the edge server;

[0101] Edge server Backhaul transmission energy consumption generated by transmitting the edge model back to the central server for:

[0102]

[0103] The total energy consumption of a single round of federated learning is:

[0104]

[0105] in, This represents the total system energy consumption for a single round of federated learning; Indicates the total number of vehicles; This indicates the total number of edge servers; Select an indicator variable for the vehicle, when the vehicle When selected ,otherwise .

[0106] The data quality score for each vehicle was calculated using a data heterogeneity assessment model. After responding to the federated learning request issued by the edge server, the edge server will distribute the global category distribution issued by the central server to the vehicles. .vehicle Based on the global category distribution and the category distribution of its own local data, the following approach is adopted: Divergence calculation data quality score ;

[0107] Configure dual disconnection detection rules; determine vehicle disconnection when any of the following conditions are met. In the Wheel disconnected:

[0108] (1) vehicles In the The total training latency of the vehicle is greater than the predicted remaining dwell time of the vehicle within the coverage area of ​​its home edge server; the dwell time of the vehicle within the wireless coverage area of ​​the roadside unit where the edge server is located is a key constraint for determining whether it can successfully complete local training and model upload. Continuing from its current position and speed, predict its location at the edge server. Remaining stay within the coverage area This typically assumes the vehicle maintains a constant current speed or uses a uniform acceleration model for short-term prediction. Based on the vehicle's future trajectory sequence, it locates the point where the distance between the vehicle's position and the edge server first exceeds the coverage radius. The time remaining is the difference between that time and the current time.

[0109] (2) vehicles Its home edge server Uplink signal-to-interference-plus-noise ratio Below the preset signal-to-interference-plus-noise ratio threshold .

[0110] The local model output by a vehicle that is determined to be offline does not participate in the current round of model aggregation. This mechanism does not require the vehicle to upload the entire driving path in advance; it only requires the edge server to make a short-term prediction based on the vehicle's currently reported motion state.

[0111] S2: Construct an optimization problem for joint vehicle selection and transmit power allocation; the optimization objective of the optimization problem is to minimize the weighted total cost, which consists of the total latency of a single round of federated learning, the total energy consumption of the system, and the data utility of successfully uploaded samples; the optimization variables of the optimization problem include a subset of vehicles to be selected and the transmit power vector of the corresponding selected vehicles;

[0112] Preferably, the optimization problem of joint vehicle selection and resource allocation includes:

[0113]

[0114]

[0115] in, Indicates the first Federated learning edge server The selected set of vehicles; Indicates the first Federated learning edge server The transmit power vector assigned to the selected vehicle; For the first Total latency of round-fed learning; For the first Total energy consumption of the federated learning system; For the first The data utility function of the successfully uploaded sample in the round of federated learning; , and This represents the weight parameter, and the weight coefficient is set to... ; Indicates the first The set of all vehicles that successfully uploaded model parameters in the Federated Learning round; For vehicles Number of samples in the local dataset; The target data volume is preset; This indicates taking the minimum value.

[0116] Based on the established hierarchical federated learning system model, the resource allocation problem in vehicular network federated learning is further formalized into a joint optimization problem: minimizing the total latency of a single round of federated learning. Total system energy consumption And the data utility of successfully uploaded samples The weighted total cost, comprised of three parts, serves as the optimization objective. Data utility is defined as the cutoff value of the ratio of successfully uploaded data to the preset target data volume. The optimization variable is the subset of vehicles to be selected for each edge server. and the corresponding transmit power vector of the selected vehicle This transforms the original problem into a mixed-integer nonlinear programming problem containing discrete variables (vehicle selection) and continuous variables (transmission power allocation), laying the foundation for subsequent solutions using a reinforcement learning framework.

[0117] S3: The optimization problem is transformed into a Markov decision process, and the SAC agent is initialized. The state space and continuous action space are defined. The action output of the SAC agent is processed using exploratory annealing and action smoothing mechanisms to obtain the processed action output. The comprehensive score of the candidate vehicles is calculated based on the processed action output. The vehicles participating in the training round are selected based on the comprehensive score. The transmission power of the selected vehicles is obtained by mapping the processed action output. A reward function is constructed. The reward function calculates the reward in real time by comprehensively considering the total latency of a single round of federated learning, the disconnection rate of the selected training vehicles, the total system energy consumption, and the data utility of successfully uploaded samples.

[0118] Preferably, the state space is a composite state vector, which is formed by concatenating the original state feature vector and the differential state feature vector; the original state feature vector is used to characterize the real-time state snapshot of the system in the current round; the differential state feature vector is obtained by subtracting the original state feature vector of the previous round from the original state feature vector of the current round, and is used to characterize the dynamic change trend of the system state; the action space is a 7-dimensional continuous action space; the first 6 dimensions of the action vector of the action space are used to calculate the weight coefficients of the comprehensive score of the candidate vehicles, and the 7th dimension of the action vector is the transmission power adjustment factor.

[0119] Preferably, the calculation of the comprehensive score for candidate vehicles includes:

[0120]

[0121] in, Indicates the first Federal learning vehicles Overall score; These are the weighting coefficients for the first 6 dimensions of the processed action output; Indicates the first Federal learning vehicles Normalized information about age; For the first Federal learning vehicles Normalized data volume; For the first Federal learning vehicles Normalized predicted dwell time; For the first Federal learning vehicles Normalized channel gain; No. Federal learning vehicles Normalized data quality score;

[0122] Based on the overall scores of each candidate vehicle, the one with the highest overall score is selected. One vehicle as an edge server Selected vehicle;

[0123] The process of obtaining the transmission power of the selected vehicle based on the processed action output mapping includes:

[0124]

[0125] in, Indicates the selected vehicle The transmission power; The 7th dimension of the output is the transmit power adjustment factor after processing.

[0126] Preferably, the process of using exploratory annealing and action smoothing mechanisms to process the action output of the SAC agent includes:

[0127] The exploratory annealing mechanism includes: setting the total number of training rounds for federated learning to [number missing]. Explore round threshold , To preset the exploration ratio; during the exploration period The mean of the Actor network's output is Gaussian sampled to generate the original action; during the utilization period, the mean of the Actor network's output is directly used as the original action.

[0128] The motion smoothing mechanism includes:

[0129]

[0130] in, Indicates the first The action output after processing by the federated learning round; The preset smoothing coefficient; For the first The federated learning process generates the original actions output by the SAC agent. The output is the action after the previous round of processing; It is the hyperbolic tangent activation function.

[0131] Preferably, the reward function includes:

[0132]

[0133]

[0134]

[0135]

[0136] in, Indicates the first Instant rewards for rounds of federated learning; , , and As a reward weight; Represents the delay penalty function; Indicates the first The disconnection rate of selected training vehicles in the round of federated learning; Represents the energy consumption penalty function; Indicates the first Number of vehicles that went offline during a round of federal learning; Indicates the first The number of training vehicles selected in the round of federated learning; This indicates the preset maximum allowable delay per round; This indicates the preset maximum allowable energy consumption per wheel.

[0137] Preferably, the total latency of a single round of federated learning and the total energy consumption of a single round of federated learning system include:

[0138]

[0139]

[0140]

[0141] in, Indicates the first Total latency of round-fed learning; This indicates that the collection of all vehicles in the local model has been successfully uploaded; Indicates the first Total training delay for vehicles in federal learning; This indicates that the edge server set of the edge model has been successfully uploaded; M represents the total number of vehicles; M represents the total number of edge servers. This indicates taking the maximum value; Indicates the first The edge aggregation latency of federated learning edge servers aggregating local models uploaded by each vehicle; Indicates the first The latency of the federated learning edge server m sending the aggregated edge model back to the central server; This represents the global aggregation latency of the central server aggregating the upload latency of edge models from various edge servers; Indicates the first Total energy consumption of the federated learning system; This variable indicates vehicle selection; 1 indicates selected, and 0 indicates unselected. Indicates the first Federal learning vehicles Total energy consumption; Indicates the first Federated learning edge server The energy consumption of edge aggregation computation generated by performing weighted aggregation operations on the local model uploaded by the vehicle; Indicates the first Federated learning edge server The energy consumption of backhaul transmission generated when transmitting the edge model back to the central server; Indicates the first Federal learning vehicles The computational energy consumed during local model training; Indicates the first Federal learning vehicles The energy consumption of transmission communication generated when uploading the local model to the edge server; Indicates vehicle The energy consumption of receiving the global model generated by the receiving edge server.

[0142] In this embodiment, the mixed-integer nonlinear programming optimization problem is transformed into a Markov decision process (MDP) and solved online using the soft actor-commentator (SAC) reinforcement learning algorithm, as detailed below:

[0143] Markov Decision Process Modeling: The optimization problem is modeled as an MDP, which consists of a quadruple of state space, action space, state transition probabilities, and reward function.

[0144] State space design: The state space is a 28-dimensional composite state vector. , consisting of a 14-dimensional original state feature vector With 14-dimensional difference state feature vector Assembly composition:

[0145]

[0146] Among them, symbols This represents a vector concatenation operation. The innovative significance of this composite state design lies in the fact that the original state features provide a "snapshot" of the system at the current moment, while the differential state features capture the direction and magnitude of state changes between two consecutive time slots. The combination of the two enables the SAC agent to perceive the dynamic evolution trend of the system. Initialized as a zero vector; original state feature vector The dimensions are defined as follows (14 dimensions in total):

[0147] : Indicates the average normalized dwell time of candidate vehicles:

[0148]

[0149] in, For the first A set of candidate vehicles for the wheel; For vehicles The predicted remaining dwell time within its own server. This is a normalized reference upper limit. This dimension reflects the average available service duration for all vehicles within the coverage area.

[0150] : Standard deviation of candidate vehicle dwell time:

[0151]

[0152] This dimension represents the standard deviation of dwell time, reflecting the degree of difference in available service time among vehicles. A larger standard deviation indicates a more uneven distribution of dwell time among vehicles, necessitating a differentiated selection strategy.

[0153] : Indicates the minimum normalized dwell time of the candidate vehicle:

[0154]

[0155] This dimension represents the minimum normalized dwell time, reflecting the time constraints of the most urgent vehicles. When this value is small, the system faces an imminent risk of disconnection.

[0156] : Represents the average normalized distance of candidate vehicles:

[0157]

[0158] in, For vehicles The current edge server index to which it belongs. This represents the three-dimensional distance from the vehicle to the edge server, reflecting the average relative distance between the vehicles.

[0159] : Standard deviation of candidate vehicle distances:

[0160]

[0161] This dimension represents the standard deviation of the distance, reflecting the degree of dispersion in the spatial distribution of vehicles.

[0162] : Indicates the maximum normalized distance of the candidate vehicles:

[0163]

[0164] This dimension represents the maximum normalized distance, reflecting the utilization of edge coverage.

[0165] : Represents the average normalized channel gain of candidate vehicles:

[0166]

[0167] in, This represents the distance from vehicle n to its home edge server. Channel gain, This represents the scaling factor, used to map the original channel gain to the [0,1] interval; this dimension reflects the average channel quality.

[0168] : Represents the standard deviation of the channel gain for candidate vehicles:

[0169]

[0170] This dimension represents the standard deviation of the channel gain, reflecting the degree of fluctuation in channel quality.

[0171] : Represents the minimum normalized channel gain of the candidate vehicle:

[0172]

[0173] This dimension represents the minimum channel gain, reflecting the bottleneck constraint of the worst-case link.

[0174] : Represents the exponential moving average of historical single-round latency:

[0175]

[0176]

[0177] in, It is the exponential moving average of historical single-round delays; ; Indicates the smoothing coefficient; The upper limit of the normalization reference is used; the advantage of using the exponential moving average instead of the simple average is that it can smooth out single-cycle fluctuations and give more weight to recent data, making the state representation more sensitive to the changing trend of the system.

[0178] : Represents the exponential moving average of historical single-round energy consumption:

[0179]

[0180]

[0181] in, This is the exponential moving average of the total energy consumption of a single-round federated learning system in history; ; Indicates the smoothing coefficient; This is the upper limit for normalization reference.

[0182] : Indicates the average normalized age of candidate vehicles:

[0183]

[0184] in, For vehicles In the The information age of a cycle is the difference between the current cycle and the cycle in which the state information was last updated. Let n be the update round corresponding to the most recent current state information received by vehicle n from any edge server. This serves as the upper limit for normalization reference.

[0185] : Indicates the maximum normalized age of the candidate vehicle:

[0186]

[0187] : Indicates reserved dimensions.

[0188] Differential state feature vector Defined as the difference between the current initial state and the previous initial state:

[0189]

[0190] In traditional reinforcement learning, the Markov property assumes that the current state contains all the information needed for decision-making. However, in highly dynamic environments like connected vehicles, the absolute value of the state is insufficient to fully reflect the system's evolution direction. For example, when the average channel gain is 0.5, if this value is rapidly decreasing (negative differential feature), it indicates that communication conditions are about to deteriorate, and a conservative strategy should be adopted in advance; conversely, if it is increasing, an aggressive strategy can be adopted. The introduction of state differential features gives the state space a "velocity" dimension, allowing the SAC agent to simultaneously perceive "where it is now" and "where it is going," significantly improving the foresight of decision-making and its adaptability to dynamic environments.

[0191] The action space is a 7-dimensional continuous action space. Among them, the first 6 dimensions The weighting coefficients used to calculate the overall score of candidate vehicles, the 7th dimension. This is the transmit power adjustment factor. The Actor network outputs a 7-dimensional diagonal Gaussian distribution. ,in, It is the mean vector. This is the standard deviation vector. The standard deviation is generated as follows:

[0192]

[0193] Among them, the clipping range of the logarithmic standard deviation. , .

[0194] The total number of training rounds for federated learning is set to... The threshold for the number of exploration rounds is Defined as:

[0195]

[0196] in, To explore the proportions, this embodiment , During the exploration phase, the system uses random sampling to generate initial actions:

[0197]

[0198] Here, ⊙ represents element-wise multiplication. for A 3D identity matrix. The advantage of reparameterized sampling is that it makes the sampling process differentiable, thus allowing gradients to flow from the Critic network to the Actor network. This is one of the key techniques enabling the SAC algorithm to train efficiently.

[0199] During the utilization period The system directly outputs the deterministic mean action:

[0200]

[0201] The innovative significance of the exploration annealing mechanism lies in the fact that: in the early stage of training, the policy is not yet mature and needs sufficient random exploration to cover the state-action space to avoid getting trapped in local optima; in the later stage of training, the policy is close to convergence, and turning off exploration noise can make the policy execute stably and avoid the impact of random perturbations on system performance.

[0202] To further suppress policy oscillations, this invention designs an action smoothing mechanism. The original actions are processed using an exponential moving average to obtain the final action vector:

[0203]

[0204] in, Using the hyperbolic tangent activation function, the action value is constrained to... interval; This is the smoothing coefficient. The formula ensures that the currently executed action is a weighted average of the original action in this round (constrained by tanh) and the action executed in the previous round, with historical actions accounting for 80% and the current action accounting for 20%, ensuring that action changes have inertia and avoiding abrupt changes. Smoothing coefficient The selection was optimized through experiments: too large a value would result in insufficient smoothing effect, while too small a value would cause the strategy response to be sluggish. It is a zero vector.

[0205] Another important function of the action smoothing mechanism is to alleviate the "action jitter" problem commonly found in reinforcement learning—where policies output significantly different actions under similar conditions. By forcing a smooth transition between consecutive actions, this invention effectively suppresses action jitter and improves the stability and predictability of the system.

[0206] Action execution: The system parses action vectors. The first six dimensions serve as vehicle selection weighting coefficients, and the seventh dimension serves as a resource allocation factor. For each vehicle n, its comprehensive score is calculated using the following formula:

[0207]

[0208] in, Let the normalized information about vehicle n be its age. Let n be the normalized data volume for vehicle n. Let n be the normalized estimated dwell time of vehicle n. Let n be the normalized channel gain for vehicle n. This represents the normalized data quality score for vehicle n. These are the weight elements corresponding to the action vector. The edge server sorts the candidate vehicles that have responded to the federated learning request and completed parameter uploading in descending order based on their comprehensive scores, selecting the top... Several high-scoring vehicles participated in this round of training. Among them, The number of available resource blocks on edge server m and the preset maximum number of participating vehicles per round are jointly determined. Vehicles use their reported available CPU frequencies for local training. For selected vehicles, the 7th dimension of the action vector is uniformly assigned a transmission power for this round, determined by the following formula:

[0209]

[0210] The transmit power ranges from 0.1W to 0.5W. The vehicle's CPU frequency uses the available value reported when responding to federated learning requests, and is not allocated by the SAC agent.

[0211] Reward Calculation: After a round of federated learning is completed, the system calculates an immediate reward based on the results of this round. The reward function comprehensively considers four dimensions: data volume utility, total latency, system energy consumption, and failure rate.

[0212]

[0213] in, This represents the total amount of data successfully uploaded in round l. The disconnection rate in round l, with each weight set as follows: .

[0214] Delay penalty function:

[0215]

[0216] Energy consumption penalty function:

[0217]

[0218] Definition of disconnection rate:

[0219]

[0220] in, This represents the number of vehicles that went offline in this round. This represents the number of vehicles selected in this round.

[0221] Initialize the SAC agent's Actor network, dual Critic network, and target network. After each round of federated learning, generate the state transition tuples from that round's interactions. Store the data in the experience replay buffer, randomly sample a batch of samples from the buffer, and update the parameters of the Actor network, the dual Critic network, and the target network.

[0222] In the iterative training phase, each round utilizes the hierarchical federated learning system model obtained in the pre-training preparation phase, the dual disconnection judgment rule, and the initialized SAC agent to perform the following steps:

[0223] S4: The central server distributes the global model to each edge server, and the edge server distributes federated learning requests to the vehicles within its coverage area; the vehicles that receive the requests and respond report their own status information according to the hierarchical federated learning system model, and the edge server generates a candidate vehicle set.

[0224] S5: Call the SAC agent to output actions based on the candidate vehicle set, obtain the selected vehicle set and the corresponding vehicle's transmission power; the selected vehicles perform local training using local data; filter vehicles for disconnection according to the dual disconnection judgment rule; the edge server performs weighted aggregation on the valid local models uploaded by the non-disconnected vehicles to obtain the edge model, and sends the edge model back to the central server; the central server receives the edge models sent back by each edge server, performs federated average aggregation, and obtains the updated global model;

[0225] Preferably, the edge server performs weighted aggregation of valid local models uploaded by vehicles that have not lost connection to obtain the edge model, including:

[0226]

[0227] in, Indicates the first Federated learning edge server The edge model obtained by aggregation; Indicates the first Federated learning edge server The set of vehicles within the coverage area that successfully uploaded their local models in this round; Indicates the first Federal learning vehicles Uploaded local model parameters; For vehicles Number of samples in the local dataset;

[0228] The central server receives the edge models returned by each edge server and performs federated average aggregation, including:

[0229]

[0230]

[0231]

[0232] in, Indicates the first The global model parameters obtained by round-fed learning aggregation; Indicates the first The federated learning process successfully transmitted the edge server set of the edge model back to the central server. Represents edge server The total number of samples from all vehicles whose models were successfully uploaded; Edge server The total number of samples from all vehicles whose models were successfully uploaded; Indicates the first Federated learning edge server The set of vehicles within the coverage area that successfully uploaded their local models in this round.

[0233] S6: Store the state transition tuple generated in this round of interaction into the experience replay buffer, sample a batch of samples from the experience replay buffer, and update the parameters of the Actor network, dual Critic network, and target network of the SAC agent.

[0234] S7: Determine whether the current communication round has reached the preset global training round; if the preset global training round has not been reached, return to step S4 to continue iterative execution; if the preset global training round has been reached, end the training and output the converged global model.

[0235] S3. The optimization problem is transformed into a Markov decision process, and the soft actor-commentator algorithm is used to solve it to obtain the optimal vehicle selection and resource allocation strategy.

[0236] In this embodiment, the SAC agent is trained based on experience-based learning: the state transition tuples of the current round are stored in an experience replay buffer with a capacity of 20,000. Each round, the SAC agent randomly samples 256 data points from the buffer for two network updates. The Critic network uses a double-Q network structure to mitigate value overestimation, with mean squared error as the loss function; the Actor network updates with the goal of maximizing the sum of state-action value and entropy regularization. The target network uses a soft update method, updating coefficients... Temperature coefficient Discount factor .

[0237] The simulation experiment in this embodiment is based on the CIFAR-10 image classification dataset. The CIFAR-10 dataset contains 60,000 32×32 color images, divided into 10 categories, with 50,000 images used for training and 10,000 images used for testing. To simulate the non-independent and identically distributed characteristics of data in a connected vehicle environment, a Dirichlet distribution is used to partition the training data, with the Dirichlet concentration parameter set to 0.5 to ensure significant class imbalance in the local data of each vehicle. Approximately 2,500 training samples are allocated to each of the 20 vehicles.

[0238] The model employs a convolutional neural network with the following structure: The first convolutional layer has a 3×32×32 input, six 5×5 convolutional kernels, ReLU activation, and 2×2 max pooling; the second convolutional layer has six input channels, sixteen 5×5 convolutional kernels, ReLU activation, and 2×2 max pooling; the first fully connected layer has 16×5×5=400-dimensional input and 120-dimensional output, with ReLU activation; the second fully connected layer has 120-dimensional input and 84-dimensional output, with ReLU activation; and the output layer has 84-dimensional input and 10-dimensional output. The model has approximately 62,000 parameters in total.

[0239] To verify the effectiveness of the proposed method, it is compared with the following three baseline methods: (1) FedAvg: a traditional federated averaging algorithm that randomly selects participating vehicles in each round and assigns a fixed CPU frequency of 1.5GHz and a fixed transmit power of 0.3W to all selected vehicles; (2) DDPG: a deep deterministic policy gradient algorithm is used for vehicle selection and resource allocation. The state space and action space are the same as the SAC method, but it does not include the double Q network and entropy regularization mechanism; (3) DDPG_VSRA: under the DDPG framework, vehicle selection adopts a scoring mechanism based on heuristic formulas, and resource allocation is still decided by the DDPG agent.

[0240] Figure 2The graph shows the model loss values ​​of four algorithms during the training process as a function of communication rounds. As can be seen from the graph, the SAC method proposed in this invention exhibits the fastest loss reduction rate in the early stages of training and maintains a lower loss level in the later stages. At the end of 1000 training rounds, the training loss of the SAC method is approximately 0.99, while the training losses of the DDPG, DDPG_VSRA, and FedAvg methods are approximately 1.25, 1.20, and 1.34, respectively. This demonstrates that this invention effectively accelerates model convergence and improves training stability through intelligent vehicle selection and resource allocation.

[0241] Figure 3 This demonstrates how the data utility of four algorithms changes with each communication round. Data utility is defined as the total number of data samples successfully uploaded and included in the aggregation in each round. From Figure 3 As can be seen, the SAC method maintained the highest data utility level throughout the training process, averaging approximately 9200 samples / round, while the data utilities of the DDPG, DDPG_VSRA, and FedAvg methods were approximately 8600, 6800, and 3500 samples / round, respectively. This result demonstrates that the multi-dimensional state-aware vehicle selection strategy of this invention can more effectively screen high-value vehicles for training.

[0242] Figure 4 The curves showing the drop rate of four algorithms as a function of communication rounds are presented. The results indicate that the SAC method maintains a consistently low drop rate, approximately 5% around 1000 rounds, while the drop rates of DDPG, DDPG_VSRA, and FedAvg methods are approximately 14%, 42%, and 48%, respectively. This invention performs local training based on the available CPU frequency reported by the vehicles and dynamically optimizes vehicle selection and transmission power, enabling vehicles to complete training and uploading tasks within a limited dwell time, effectively reducing drop events caused by timeouts.

[0243] Figure 5 The curves showing the change of immediate reward for four algorithms with the number of communication rounds are presented. The reward value of the SAC method rises rapidly from an initial negative value, stabilizing between 2.0 and 2.3 after approximately 500 rounds. The reward value of the DDPG method is approximately 1.5 to 1.6, the reward value of the DDPG_VSRA method is approximately 0.4 to 0.45, and the reward value of the FedAvg method is approximately 0. The reward value of the SAC method is significantly higher than that of the other three methods, indicating that the joint optimization strategy proposed in this invention has the best overall performance in multi-objective scenarios that comprehensively consider data volume, latency, and disconnection rate.

[0244] Figure 6The test accuracy of four algorithms is compared as latency and energy consumption increase. Cumulative system latency and energy consumption are defined as the total accumulated latency and energy consumption from the start of training to the current round. Under the same cumulative latency and energy consumption, the SAC method achieves higher test accuracy. At convergence, the SAC method achieves a test accuracy of approximately 63%, while the DDPG, DDPG_VSRA, and FedAvg methods achieve 60%, 59%, and 57%, respectively. This demonstrates that the present invention has a significant advantage in latency efficiency, i.e., it achieves superior model performance with the same time cost.

[0245] Based on the experimental results above, the proposed vehicular network resource allocation method based on federated learning and SAC reinforcement learning significantly outperforms existing baseline methods in terms of model convergence speed, data utility, disconnection rate control, and comprehensive reward. This is mainly attributed to the following three aspects: First, the multi-dimensional state vector design enables the SAC agent to comprehensively perceive vehicle status, channel conditions, and system historical information; second, the exploratory annealing and action smoothing mechanisms ensure the stability of policy convergence; and third, the vehicle selection strategy based on comprehensive scoring effectively identifies high-value vehicles.

[0246] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A resource allocation method for vehicle-to-everything (V2X) networks based on federated learning, characterized in that, The method is applied to a layered vehicular network federated learning architecture consisting of vehicles, edge servers, and central servers, and includes a pre-training preparation phase and a cyclic iterative training phase. The following steps should be performed during the training preparation phase: S1: Construct a hierarchical federated learning system model; the constructed hierarchical federated learning system model includes a communication model, a latency model, an energy consumption model, and a data heterogeneity evaluation model; calculate the data quality score of each vehicle through the data heterogeneity evaluation model, and configure a dual disconnection judgment rule, the local model output by the vehicle that is judged to be disconnected will not participate in the model aggregation in this round; S2: Construct an optimization problem for joint vehicle selection and transmit power allocation; the optimization objective of the optimization problem is to minimize the weighted total cost, which consists of the total latency of a single round of federated learning, the total energy consumption of the system, and the data utility of successfully uploaded samples; the optimization variables of the optimization problem include a subset of vehicles to be selected and the transmit power vector of the corresponding selected vehicles; S3: The optimization problem is transformed into a Markov decision process, and the SAC agent is initialized. The state space and continuous action space are defined. The action output of the SAC agent is processed using exploratory annealing and action smoothing mechanisms to obtain the processed action output. The comprehensive score of the candidate vehicles is calculated based on the processed action output. The vehicles participating in the training round are selected based on the comprehensive score. The transmission power of the selected vehicles is obtained by mapping the processed action output. A reward function is constructed. The reward function calculates the reward in real time by comprehensively considering the total latency of a single round of federated learning, the disconnection rate of the selected training vehicles, the total system energy consumption, and the data utility of successfully uploaded samples. In the iterative training phase, each round utilizes the hierarchical federated learning system model obtained in the pre-training preparation phase, the dual disconnection judgment rule, and the initialized SAC agent to perform the following steps: S4: The central server distributes the global model to each edge server, and the edge server distributes federated learning requests to the vehicles within its coverage area; the vehicles that receive the requests and respond report their own status information according to the hierarchical federated learning system model, and the edge server generates a candidate vehicle set. S5: Call the SAC agent to output actions based on the candidate vehicle set, and obtain the selected vehicle set and the corresponding vehicle's transmission power in this round; The selected vehicles perform local training using local data; the vehicles are filtered for disconnection according to the dual disconnection judgment rule; the edge server performs weighted aggregation of the valid local models uploaded by the vehicles that have not disconnected to obtain the edge model, and sends the edge model back to the central server; the central server receives the edge models sent back by each edge server, performs federated average aggregation, and obtains the updated global model. S6: Store the state transition tuple generated in this round of interaction into the experience replay buffer, sample a batch of samples from the experience replay buffer, and update the parameters of the Actor network, dual Critic network, and target network of the SAC agent. S7: Determine whether the current communication round has reached the preset global training round; if the preset global training round has not been reached, return to step S4 to continue iterative execution; if the preset global training round has been reached, end the training and output the converged global model.

2. The method for allocating vehicle network resources based on federated learning according to claim 1, characterized in that, The communication model adopts the orthogonal frequency division multiple access (OFDMA) communication method to calculate the channel gain between the vehicle and the edge server, the uplink signal-to-interference-plus-noise ratio (SINORR) of the vehicle uploading the local model to the edge server, and the corresponding uplink transmission rate. The latency model is used to calculate the following latency: vehicle local model training latency, vehicle receiving latency of the global model sent by the edge server, vehicle uploading latency of the trained local model to the edge server, edge aggregation latency of the edge server aggregating the local models uploaded by each vehicle, edge server backhaul latency of the aggregated edge model to the central server, and central server aggregating the edge models uploaded by each edge server. The total vehicle training latency is the sum of the vehicle local model training latency, the vehicle receiving latency of the global model sent by the edge server, and the vehicle uploading latency of the trained local model to the edge server. The energy consumption model is used to calculate the computational energy consumption generated by the vehicle performing local model training, the transmission communication energy consumption generated by the vehicle uploading the local model to the edge server, the receiving energy consumption generated by the vehicle receiving the global model sent by the edge server, the edge aggregation computational energy consumption generated by the edge server performing weighted aggregation operation on the local model uploaded by the vehicle, and the return transmission energy consumption generated by the edge server returning the edge model to the central server. The dual disconnection determination rule includes: when the total training latency of the vehicle is greater than the predicted remaining dwell time of the vehicle within the coverage area of ​​its home edge server, or when the uplink signal-to-interference-plus-noise ratio between the vehicle and its home edge server is lower than a preset signal-to-interference-plus-noise ratio threshold, the vehicle is determined to be disconnected; the local model output by the vehicle determined to be disconnected does not participate in the current round of model aggregation.

3. The method for allocating vehicle network resources based on federated learning according to claim 2, characterized in that, The data heterogeneity assessment model uses KL divergence to calculate the vehicle's data quality score, including: vehicle After responding to the federated learning request issued by the edge server, the edge server will distribute the global category distribution issued by the central server to the vehicles. ;vehicle Calculate vehicle categories based on global category distribution and category distribution of local data. Data quality score : in, Represents an exponential function; Indicates vehicle Category Divergence; This indicates the total number of categories in the data sample; Indicates vehicle Categories in local data samples The proportion; Represents the categories in the global data sample The percentage.

4. The vehicle network resource allocation method based on federated learning according to claim 3, characterized in that, The optimization problem of joint vehicle selection and resource allocation includes: in, Indicates the first Federated learning edge server The selected set of vehicles; Indicates the first Federated learning edge server The transmit power vector assigned to the selected vehicle; For the first Total latency of round-fed learning; For the first Total energy consumption of the federated learning system; For the first The data utility function of the successfully uploaded sample in the round of federated learning; , and Indicates the weighting parameter; Indicates the first The set of vehicles that successfully uploaded model parameters in the federated learning round; For vehicles Number of samples in the local dataset; The target data volume is preset; This indicates taking the minimum value.

5. The method for allocating vehicle network resources based on federated learning according to claim 4, characterized in that, The state space is a composite state vector, which is formed by concatenating the original state feature vector and the differential state feature vector. The original state feature vector is used to represent the real-time state snapshot of the system in the current round. The differential state feature vector is obtained by subtracting the original state feature vector of the previous round from the original state feature vector of the current round, and is used to represent the dynamic change trend of the system state. The action space is a 7-dimensional continuous action space. The first 6 dimensions of the action vector in the action space are used to calculate the weight coefficients of the comprehensive score of the candidate vehicles, and the 7th dimension of the action vector is the transmission power adjustment factor.

6. The method for allocating vehicle network resources based on federated learning according to claim 5, characterized in that, The calculation of the overall score for candidate vehicles includes: in, Indicates the first Federal learning vehicles Overall score; These are the weighting coefficients for the first 6 dimensions of the processed action output; Indicates the first Federal learning vehicles Normalized information about age; For the first Federal learning vehicles Normalized data volume; For the first Federal learning vehicles Normalized predicted dwell time; For the first Federal learning vehicles Normalized channel gain; No. Federal learning vehicles Normalized data quality score; Based on the overall scores of each candidate vehicle, the one with the highest overall score is selected. One vehicle as an edge server Selected vehicle; The process of obtaining the transmission power of the selected vehicle based on the processed action output mapping includes: in, Indicates the selected vehicle The transmission power; The 7th dimension of the output is the transmit power adjustment factor after processing.

7. The method for allocating vehicle network resources based on federated learning according to claim 6, characterized in that, The process of processing the action output of the SAC agent using exploratory annealing and action smoothing mechanisms includes: The exploratory annealing mechanism includes: setting the total number of training rounds for federated learning to [number missing]. Explore round threshold , To preset the exploration ratio; during the exploration period The mean of the Actor network's output is Gaussian sampled to generate the original action; during the utilization period, the mean of the Actor network's output is directly used as the original action. The motion smoothing mechanism includes: in, Indicates the first The action output after processing by the federated learning round; The preset smoothing coefficient; For the first The federated learning process generates the original actions output by the SAC agent. The output is the action after the previous round of processing; It is the hyperbolic tangent activation function.

8. A method for allocating vehicle network resources based on federated learning according to claim 7, characterized in that, The reward function includes: in, Indicates the first Instant rewards for rounds of federated learning; , , and As a reward weight; Represents the delay penalty function; Indicates the first The disconnection rate of selected training vehicles in the round of federated learning; Represents the energy consumption penalty function; Indicates the first Number of vehicles that went offline during a round of federal learning; Indicates the first The number of training vehicles selected in the round of federated learning; This indicates the preset maximum allowable delay per round; This indicates the preset maximum allowable energy consumption per wheel.

9. A method for allocating vehicle network resources based on federated learning according to claim 8, characterized in that, The edge server performs a weighted aggregation of valid local models uploaded by vehicles that have not lost connection to obtain edge models, including: in, Indicates the first Federated learning edge server The edge model obtained by aggregation; Indicates the first Federated learning edge server The set of vehicles within the coverage area that successfully uploaded their local models in this round; Indicates the first Federal learning vehicles Uploaded local model parameters; For vehicles Number of samples in the local dataset; The central server receives the edge models returned by each edge server and performs federated average aggregation, including: in, Indicates the first The global model parameters obtained by round-fed learning aggregation; Indicates the first The federated learning process successfully transmitted the edge server set of the edge model back to the central server. Represents edge server The total number of samples from all vehicles whose models were successfully uploaded; Edge server The total number of samples from all vehicles whose models were successfully uploaded; Indicates the first Federated learning edge server The set of vehicles within the coverage area that successfully uploaded their local models in this round.

10. A method for allocating vehicle network resources based on federated learning according to claim 9, characterized in that, The total latency and total energy consumption of a single-round federated learning system include: in, Indicates the first Total latency of round-fed learning; This indicates that the collection of all vehicles in the local model has been successfully uploaded; Indicates the first Total training delay for vehicles in federal learning; This indicates that the edge server set of the edge model has been successfully uploaded; M represents the total number of vehicles; M represents the total number of edge servers. This indicates taking the maximum value; Indicates the first The edge aggregation latency of federated learning edge servers aggregating local models uploaded by each vehicle; Indicates the first The latency of the federated learning edge server m sending the aggregated edge model back to the central server; This represents the global aggregation latency of the central server aggregating the upload latency of edge models from various edge servers; Indicates the first Total energy consumption of the federated learning system; This variable indicates vehicle selection; 1 indicates selected, and 0 indicates unselected. Indicates the first Federal learning vehicles Total energy consumption; Indicates the first Federated learning edge server The energy consumption of edge aggregation computation generated by performing weighted aggregation operations on the local model uploaded by the vehicle; Indicates the first Federated learning edge server The energy consumption of backhaul transmission generated when transmitting the edge model back to the central server; Indicates the first Federal learning vehicles The computational energy consumed during local model training; Indicates the first Federal learning vehicles The energy consumption of transmission communication generated when uploading the local model to the edge server; Indicates vehicle The energy consumption of receiving the global model generated by the receiving edge server.