Task unloading and resource allocation method and system based on federal self-supervised learning

Abstract

The invention provides a task unloading and resource allocation method and system based on federal self-supervised learning, and belongs to the technical field of edge computing of the Internet of Vehicles. The method comprises the following steps: constructing a first local model of a target vehicle; determining the number of task unloading iterations and the number of non-training iterations; determining a second local model of the target vehicle; unloading the local data to the RSU to determine a third local model; determining a system state of the target vehicle, an action of the target time slot, and a reward to update the SAC network; determining a global model of the base station; resource allocation is carried out according to the final SAC network, and task unloading is carried out according to the final global model. According to the method, the system energy consumption is reduced, and the unloading efficiency and accuracy of federal self-supervised learning are improved.

Description

Technical Field

[0001] The present invention belongs to the field of edge computing technology for Internet of Vehicles, and in particular, relates to a method and system for task offloading and resource allocation based on federated self-supervised learning. Background Art

[0002] With the rapid popularization of Internet of Things (IoT) devices and the growing demand for intelligent transportation systems, the Internet of Vehicle (IoV) has become an important part of modern transportation systems. Vehicle Edge Computing (VEC) solves this problem by deploying computing resources at the edge of the network, especially at the Road Side Unit (RSU).

[0003] Self-supervised learning can utilize a large amount of unlabeled image data to achieve image recognition and classification in the field of autonomous driving. Through self-supervised learning methods, the system can automatically learn effective feature representations from unlabeled data, reduce dependence on manual labeling, and thus reduce data labeling costs. Self-supervised learning is particularly critical in complex environment perception tasks, helping to improve the recognition accuracy and generalization ability of autonomous driving systems for a variety of scenarios, and providing important support for enhancing the safety and reliability of vehicles in actual road environments. At the same time, this also significantly increases the computing load and resource consumption in IoT scenarios.

[0004] Resource allocation and task offloading are important application scenarios of VEC. By offloading some computing tasks to edge servers, vehicles can effectively reduce their own computing load, optimize resource utilization, and improve computing efficiency without increasing latency. This approach is particularly important in environments with low latency and high performance requirements, such as real-time decision-making and complex driver assistance systems. In addition, resource allocation algorithms can dynamically adjust edge computing resources to adapt to changes in vehicle mobility and network conditions, thereby ensuring system stability and service quality.

[0005] Existing technologies usually use federated self-supervised learning for image classification during autonomous driving to protect vehicle privacy. However, these methods often do not fully consider resource allocation issues and ignore whether the vehicle has the ability to complete all local training tasks in each training cycle. Such limitations may lead to inefficient resource utilization and affect the overall performance of the system when computing and storage resources are limited. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a method and system for task offloading and resource allocation based on federated self-supervised learning.

[0007] In a first aspect, the present invention provides a method for task offloading and resource allocation based on federated self-supervised learning, comprising: S1, obtaining a global model of federated training stored in a base station and a first transmission power, CPU frequency, and task offloading ratio allocated by the base station to a target vehicle; S2, constructing a first local model of the target vehicle according to each layer parameter and model structure of the global model; S3, determining the number of task offloading iterations and the number of untrained iterations according to the CPU frequency and the task offloading ratio; S4, determining a new local model of the target vehicle according to the first local model and the local data to serve as a second local model; S5, unloading the local data to the RSU at the first transmission power according to the number of task offloading iterations, so as to determine a local model of the RSU and use it as a third local model; S6, uploading the second local model, the third local model, the location information of the target vehicle, the speed of the target vehicle in the target time slot, the total energy consumption of the target vehicle, and the number of untrained iterations to the base station to determine the system state of the target vehicle, the action and the reward of the target time slot; S7, updating the SAC network according to the system state of the target vehicle, the action and reward of the target time slot; S8, determining a global model of the base station according to the second local models and the third local models corresponding to all vehicles participating in the federated training; S9, repeating the operations of S1-S8 until the preset training rounds of federated training are reached, and the final SAC network and the global model are obtained, so as to allocate resources according to the final SAC network and offload tasks according to the final global model.

[0008] Optionally, determining the number of task offloading iterations and the number of untrained iterations according to the CPU frequency and the task offloading ratio includes: Get the total number of training iterations required for vehicle n in time slot t ; Determine the expected number of training iterations for the local model And the actual number of training iterations of the local model ; Determine the total number of training iterations N on the RSU side R ; The expected number of task offloading training iterations on the RSU side is calculated according to the following formula : ; in, is a binary variable, indicating whether vehicle n needs to offload part of its tasks to the RSU; when the task offloading ratio of vehicle n When it is less than the uninstall ratio threshold, ; When the task unloading ratio of vehicle n When the unloading ratio is not less than the threshold, ; Will and The minimum value among them is taken as the number of task offloading training iterations ; The number of untrained iterations is calculated according to the following formula : .

[0009] Optionally, determining a new local model of the target vehicle according to the first local model and the local data as the second local model includes: The information-noise contrast estimation loss is calculated according to the following formula : ; Among them, exp(·) represents an exponential function with a natural constant as the base; For enhanced local data The anchor point sample is input to the function χ which consists of an encoder, two fully connected neural networks and a ReLU function; For enhanced local data The positive sample output after input into the function χ; For enhanced local data The negative sample output after being input to the function χ; τ is the hyperparameter of temperature; K is the total number of local data marked as negative samples; i≠j; π1(·) is the first data enhancement function; π2(·) is the second data enhancement function; The loss of dual temperature is calculated according to the following formula : ; Among them, sg[·] represents the stop gradient; The information-noise contrast estimation loss when the temperature hyperparameter is τ2; The information-noise contrast estimation loss when the temperature hyperparameter is τ1; Constructing a second local model based on the loss of dual temperature expression: ; in, represents the first local model; η r is the learning rate; is the gradient descent algorithm; Z is the total number of local data.

[0010] Optionally, uploading the second local model, the third local model, the location information of the target vehicle, the speed of the target vehicle in the target time slot, the total energy consumption of the target vehicle, and the number of untrained iterations to the base station to determine the global model of the base station, the system state of the target vehicle, and the action and reward of the target time slot includes: The number of iterations of local model training is calculated according to the following formula : ; The total energy consumption of each vehicle at time slot t is calculated according to the following formula: ; in, is the total energy consumption of vehicle n in time slot t; is the computing energy consumption of RSU; Calculate energy consumption for local areas; is the transmission energy consumption of RSU; The energy consumed to perform one training iteration for vehicle n; The energy consumed by performing one training iteration for the RSU; Build system status t expression: ; in, is the fading information, which is determined by the current position of the vehicle; ; is the fading information of vehicle n at time slot t; is the vehicle speed; ; is the speed of vehicle n at time slot t; N is the total number of vehicles participating in the federated training; Construct action a in time slot t t expression: ; in, The power of all vehicles when sending data to RSU; ; is the transmission power of vehicle n transmitting data to RSU; The CPU frequency allocated to all vehicles by the base station; ; The CPU frequency assigned to vehicle n by the base station; The mission unloading ratio for all vehicles; ; is the task unloading ratio of vehicle n; Build Rewards t expression: ; Among them, θ1 is The weight of θ2 is the overload The weight of The weight of .

[0011] Optionally, determining the global model of the base station according to the second local model and the third local model corresponding to all vehicles participating in the federated training includes: Build a global model of base stations expression: ; in, represents the second local model; represents the third local model; N is the total number of vehicles participating in the federated training.

[0012] In a second aspect, the present invention provides a task offloading and resource allocation system based on federated self-supervised learning, comprising: An acquisition module, used to acquire a global model of federated training stored in a base station and a first transmission power, a CPU frequency, and a task offloading ratio allocated by the base station to a target vehicle; A construction module, used for constructing a first local model of the target vehicle according to each layer parameter and model structure of the global model; A first determination module is used to determine the number of task offloading iterations and the number of untrained iterations according to the CPU frequency and the task offloading ratio; A second determination module, used for determining a new local model of the target vehicle according to the first local model and the local data, to serve as a second local model; A third determination module is used to offload the local data to the RSU at a first transmission power according to the number of task offloading iterations, so as to determine a local model of the RSU as a third local model; a fourth determination module, configured to upload the second local model, the third local model, the location information of the target vehicle, the speed of the target vehicle in the target time slot, the total energy consumption of the target vehicle, and the number of untrained iterations to the base station to determine the system state of the target vehicle, the action and the reward in the target time slot; A network update module, used to update the SAC network according to the system status of the target vehicle, the action and reward of the target time slot; A fifth determination module, configured to determine a global model of the base station according to the second local models and the third local models corresponding to all vehicles participating in the federated training; A loop module is used to repeatedly execute the operations of the acquisition module to the fifth determination module until a preset training round of federated training is reached, so as to obtain a final SAC network and a global model, so as to perform resource allocation according to the final SAC network and perform task unloading according to the final global model.

[0013] Optionally, the first determining module includes: The acquisition unit is used to obtain the total number of training iterations required for vehicle n in time slot t ; The first determination unit is used to determine the estimated number of training iterations of the local model And the actual number of training iterations of the local model ; The second determination unit is used to determine the total number of training iterations N on the RSU side. R ; The first calculation unit is used to calculate the expected number of task offloading training iterations on the RSU side according to the following formula : ; in, is a binary variable, indicating whether vehicle n needs to offload part of its tasks to the RSU; when the task offloading ratio of vehicle n When it is less than the uninstall ratio threshold, ; When the task unloading ratio of vehicle n When the unloading ratio is not less than the threshold, ; The third determining unit is used to and The minimum value among them is taken as the number of task offloading training iterations ; The second calculation unit is used to calculate the number of untrained iterations according to the following formula : .

[0014] Optionally, the second determining module includes: The third calculation unit is used to calculate the information noise contrast estimation loss according to the following formula : ; Among them, exp(·) represents an exponential function with a natural constant as the base; For enhanced local data The anchor point sample is input to the function χ which consists of an encoder, two fully connected neural networks and a ReLU function; For enhanced local data The positive sample output after input into the function χ; For enhanced local data The negative sample output after being input to the function χ; τ is the hyperparameter of temperature; K is the total number of local data marked as negative samples; i≠j; π1(·) is the first data enhancement function; π2(·) is the second data enhancement function; The fourth calculation unit is used to calculate the loss of dual temperature according to the following formula : ; Among them, sg[·] represents the stop gradient; The information-noise contrast estimation loss when the temperature hyperparameter is τ2; The information-noise contrast estimation loss when the temperature hyperparameter is τ1; The first building unit is used to build a second local model according to the loss of the dual temperature expression: ; in, represents the first local model; η r is the learning rate; is the gradient descent algorithm; Z is the total number of local data.

[0015] Optionally, the fourth determining module includes: The fifth calculation unit is used to calculate the number of iterations of local model training according to the following formula : ; The sixth calculation unit is used to calculate the total energy consumption of each vehicle at time slot t according to the following formula: ; in, is the total energy consumption of vehicle n in time slot t; is the computing energy consumption of RSU; Calculate energy consumption for local areas; is the transmission energy consumption of RSU; The energy consumed to perform one training iteration for vehicle n; The energy consumed by performing one training iteration for the RSU; The second building unit is used to build the system state s t expression: ; in, is the fading information, which is determined by the current position of the vehicle; ; is the fading information of vehicle n at time slot t; is the vehicle speed; ; is the speed of vehicle n at time slot t; N is the total number of vehicles participating in the federated training; The third construction unit is used to construct an action a in time slot t t expression: ; in, The power of all vehicles when sending data to RSU; ; is the transmission power of vehicle n transmitting data to RSU; The CPU frequency allocated to all vehicles by the base station; ; The CPU frequency assigned to vehicle n by the base station; The mission unloading ratio for all vehicles; ; is the task unloading ratio of vehicle n; The fourth building block is used to build reward r t expression: ; Among them, θ1 is The weight of θ2 is the overload The weight of The weight of .

[0016] Optionally, the fifth determining module includes: The fifth construction unit is used to construct a global model of the base station expression: ; in, represents the second local model; represents the third local model; N is the total number of vehicles participating in the federated training.

[0017] The present invention provides a task offloading and resource allocation method and system based on federated self-supervised learning, in which the SAC algorithm is used to allocate the transmission power from the vehicle to the base station and the RSU, the CPU frequency in the vehicle, and the computing resource allocation ratio in the RSU. According to the allocated ratio, the computing resources available to each vehicle in the RSU are calculated. Then, according to the allowed computing resources, each local model training iteration task is divided into local training and RSU-assisted training. An offloading ratio threshold is set to optimize the task offloading efficiency. After offloading part of the iterative tasks to the RSU, the vehicle processes the remaining iterative tasks locally. If there are still unfinished iterations in a round of federated learning, the remaining number of iterations will be stored and wait for the next training, which reduces the system energy consumption and improves the offloading efficiency and accuracy of federated self-supervised learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 A flowchart of a method for task offloading and resource allocation based on federated self-supervised learning provided by an embodiment of the present invention; Figure 2 An application scenario diagram of the task offloading and resource allocation method based on federated self-supervised learning provided by an embodiment of the present invention; Figure 3 A comparison diagram of the energy consumption of the SAC algorithm provided in the embodiment of the present invention and other DRL methods; Figure 4 The embodiment of the present invention provides Figure 3 Detailed diagram of the round interval [1,500] in the figure; Figure 5 A comparison result diagram of the SAC algorithm provided in an embodiment of the present invention and other DRL methods in terms of the number of calculations; Figure 6 A comparison result diagram of the SAC algorithm provided in an embodiment of the present invention and other DRL methods in terms of reward function; Figure 7 The embodiment of the present invention provides Figure 6 Detailed diagram of the round interval [1,500] in the figure; Figure 8 The embodiment of the present invention provides Figure 6 Detailed diagram in the round interval [2700,3000]; Fig. 9 A comparison chart of the overload times of the SAC algorithm provided in the embodiment of the present invention and other DRL methods; Fig.10 A comparison result diagram of the overload rate between the SAC algorithm provided in an embodiment of the present invention and other DRL methods; Fig.11 A comparison diagram of the SAC algorithm provided in an embodiment of the present invention and other DRL methods in improving unloading efficiency; Fig.12 A comparison result diagram of energy consumption of the SAC algorithm provided by an embodiment of the present invention under different numbers of vehicles; Fig.13 A comparison result diagram of the reward function of the SAC algorithm provided by the embodiment of the present invention under different numbers of vehicles; Fig.14A comparison chart of test results of different DRL methods provided in an embodiment of the present invention; Fig.15 Some parameters and some parameter value diagrams provided for the embodiments of the present invention; Fig.16 A structural diagram of a task offloading and resource allocation system based on federated self-supervised learning provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] The present invention aims to minimize energy consumption, performs task offloading and resource allocation based on task offloading ratio threshold and deep reinforcement learning, and considers a vehicle communication network in an urban scenario, which is composed of a base station (BS), a road side unit (RSU) and N vehicles.

[0022] Example 1.

[0023] like Figure 1 As shown, this embodiment provides a task offloading and resource allocation method based on federated self-supervised learning, including: S1, obtaining the global model of federated training stored in the base station and the first transmission power, CPU frequency and task offloading ratio allocated by the base station to the target vehicle.

[0024] like Figure 2 As shown in the figure, the scenario is an urban VEC traffic system with four intersections. In this scenario, a base station is located at a corner, a roadside unit is deployed on the roadside, and there are N vehicles driving on the road. Assume that these four intersections are within the coverage of the base station and RSU. When vehicle n arrives at the intersection, it will choose to turn left, turn right, or go straight through the intersection with probabilities φ1, φ2, and φ3 respectively. The neural network configured on the base station is used to provide decision support. The RSU has fixed computing resources and can process task offloading from vehicles in parallel.

[0025] The whole process is divided into two stages: offloading-based federated self-supervised learning (SSL) and deep reinforcement learning (DRL)-based resource allocation. These two stages are independent of each other but also influence each other. max fragments, for each fragment e, , is divided into R max Similarly, the resource allocation phase based on DRL, namely SAC (Soft Actor-Critic) algorithm, also includes K max segments, each segment e is divided into S max For each round r(r∈[1,R max ]) and time slot t(t∈[1,S max ]) has a duration of T. Each round of federation corresponds to a time slot, but when the slot t reaches S max When , segment e transfers to the next segment e+1, at which time slot t is reset to zero, and rounds r continue to accumulate until the entire algorithm is completed. Therefore, the relationship between round r and slot t in segment e can be described as: .

[0026] S2, constructing a first local model of the target vehicle according to each layer parameter and model structure of the global model.

[0027] Before each time slot t starts, the base station stores the global model obtained from round r-1 aggregation and uses DRL to allocate resources, including transmission power for data transmission to RSU and base station and , CPU frequency for local training , and the task allocation ratio of vehicle n When segment e = 1, in the first round / time slot, the base station randomly initializes the parameters of the global model and the DRL network. At the same time, vehicle n stores Z images captured in the previous round, called local data, and has a model with the same structure as the global model on the base station side, called the local model.

[0028] S3, determining the number of task offloading iterations and the number of untrained iterations according to the CPU frequency and the task offloading ratio.

[0029] In each time slot t, vehicle n first downloads the stored data from the BS and distributes the parameters of the BS-side global model to the local model. Next, according to the CPU frequency of the download Calculate the theoretical total number of local training iterations for vehicle n , and calculate the actual total number of training iterations based on the available CPU time T' of vehicle n Finally, vehicle n is assigned a ratio according to the downloaded tasks (task offloading ratio) and the task allocation ratio threshold q0 (offloading ratio threshold) determine whether to offload the computing task to the RSU in time slot t. Specifically, when When q0 is greater than q0, vehicle n will offload part of the task to the RSU; otherwise, the vehicle will not offload part of the task to the RSU and Reset to 0. If the decision is made to offload the task, vehicle n will immediately pass the downloaded transmission power (First transmission power) Send an unloading request to the RSU. The request includes the local model, local data, and the task allocation ratio of vehicle n in time slot t.

[0030] In this embodiment, the total task that vehicle n needs to process in time slot t is recorded as (Total number of training iterations), which includes the estimated number of training iterations of the local model in time slot t and the remaining number of training iterations not processed from the previous time slot When vehicle n offloads part of its task to the RSU, the total task The ratio will be allocated according to the assigned tasks The number of iterations that are divided into local model training And the number of training iterations offloaded to RSU. The total number of training iterations N on the RSU side can be calculated according to the following formula R : .

[0031] Among them, t max is the maximum time used to transmit data; is the time required for vehicle n to transmit data to the RSU, i.e., the transmission delay; D is the size of the local model transmitted by each vehicle; in this embodiment, the size of the local model transmitted by each vehicle is the same; B is the information transmission rate between the vehicle and the RSU; n is the bandwidth occupied by vehicle n when transmitting data to RSU; N0 is the noise power; is the channel gain at time slot t.

[0032] The vehicles involved in the training use orthogonal frequency division multiple access (OFDM) technology and RSU communication, so the interference between vehicles will not be considered during the communication process. Each vehicle uses the same channel model in the same time slot, and the channel model will be updated as time slot t enters time slot t+1. Therefore, at time slot t, the channel gain can be expressed as: .

[0033] Where dis(R,n) is the distance between vehicle n and RSU, in meters; To represent small-scale fading, and obey the exponential distribution with unit mean, that is, ; Represents the effect of large-scale fading on the signal, including path loss and shadow fading .but It can be expressed as: .

[0034] The shadow fading It follows a normal distribution with a mean of 0 and a standard deviation of 8, expressed as , and in high-density urban environments, the path loss It can be expressed as: .

[0035] Exemplarily, this step includes: Get the total number of training iterations required for vehicle n in time slot t .

[0036] Determine the expected number of training iterations for the local model And the actual number of training iterations of the local model .

[0037] Determine the total number of training iterations N on the RSU side R .

[0038] The expected number of task offloading training iterations on the RSU side is calculated according to the following formula : .

[0039] in, is a binary variable, indicating whether vehicle n needs to offload part of its tasks to the RSU; when the task offloading ratio of vehicle n When it is less than the uninstall ratio threshold, ; When the task unloading ratio of vehicle n When the unloading ratio is not less than the threshold, .

[0040] if Beyond , which indicates that the desired offloaded task is too large and exceeds the total amount of tasks that need to be processed locally. In this case, the number of training iterations actually offloaded should be recorded as Therefore, and The minimum value among them is taken as the number of task offloading training iterations .

[0041] The number of untrained iterations is calculated according to the following formula : .

[0042] S4, determining a new local model of the target vehicle according to the first local model and the local data to serve as a second local model.

[0043] The training process is federated self-supervised learning. First, each local data Input to two different data augmentation methods π1(·) and π2(·). The remaining images is marked as a negative sample. and Input is fed into a function χ consisting of an encoder, two fully connected neural networks, and a ReLU function. The output of function χ includes anchor samples , positive sample And the encoded negative samples ; i∈[1,Z], j∈[1,Z].

[0044] Exemplarily, this step includes: The Information Noise Contrastive Estimation (InforNCE) loss is calculated according to the following formula : .

[0045] Among them, exp(·) represents an exponential function with a natural constant as the base; For enhanced local data The anchor point sample is input to the function χ which consists of an encoder, two fully connected neural networks and a ReLU function; For enhanced local data The positive sample output after input into the function χ; For enhanced local data The negative sample output after being input to the function χ; τ is the hyperparameter of temperature; K is the total number of local data marked as negative samples; i≠j; π1(·) is the first data enhancement function; π2(·) is the second data enhancement function.

[0046] The loss of dual temperature is calculated according to the following formula : .

[0047] Among them, sg[·] represents the stop gradient; The information-noise contrast estimation loss when the temperature hyperparameter is τ2; Denotes the information-noise contrastive estimation loss when the temperature hyperparameter is τ1.

[0048] Constructing a second local model based on the loss of dual temperature expression: .

[0049] in, represents the first local model; η r is the learning rate; is the gradient descent algorithm; Z is the total number of local data.

[0050] S5, according to the number of task offloading iterations, offloading the local data to the RSU at the first transmission power to determine the local model of the RSU, and use it as the third local model.

[0051] After receiving the task offloading request from the vehicle, the RSU uses the local data and local model uploaded by vehicle n to perform Training again to generate a new model (i.e. the third local model), the training process is similar to the vehicle local training process. After the training is completed, the RSU will Send back to vehicle.

[0052] S6, uploading the second local model, the third local model, the location information of the target vehicle, the speed of the target vehicle in the target time slot, the total energy consumption of the target vehicle and the number of untrained iterations to the base station to determine the system state of the target vehicle, the action and reward of the target time slot.

[0053] After receiving all the training models sent by RSU, vehicle n will use the locally trained model And the model trained by RSU , the speed of the next time slot t+1 , total energy consumption and the number of unprocessed training iterations Transmission power via download Upload to the base station.

[0054] During the training process, the energy consumed by vehicle n to perform one training iteration is calculated as follows: .

[0055] in, , and They represent the power, computational delay, and energy consumed for one iteration of training when vehicle n performs local training locally; is the vehicle's local CPU computing frequency, which can be calculated using the dynamic voltage frequency scaling (DVFS) method, namely: .

[0056] in, is the effective switched capacitance, which depends on the chip architecture, Indicates the CPU frequency assigned to the local computer, and , and Represent the maximum and minimum CPU computing frequencies respectively. To calculate the local computing delay of the vehicle, let c be the number of CPU cycles required to process unit size data, and Z be the training data size, then cZ represents the number of CPU cycles required for training. Therefore, the calculation delay The calculation is as follows: .

[0057] You can get: .

[0058] Similarly, the delay of each round of calculation on RSU can be calculated and the energy consumed : .

[0059] .

[0060] in, Indicates the CPU frequency for RSU calculation.

[0061] If the vehicle offloads part of the computing tasks to the RSU, the transmission energy consumption of vehicle n during the transmission process can be expressed as: .

[0062] Therefore, the energy consumed by vehicle n to transmit to RSU It can be expressed as: .

[0063] When a vehicle is performing a local iterative task, if it cannot complete all iterative tasks due to the limitation of local computing resources, it needs to offload some of the iterative tasks to the RSU. This offloading process requires the base station to use the DRL algorithm to allocate the task allocation ratio and determine the computing resources allowed by the RSU for each vehicle. The total iterative tasks will be allocated to local training and training on the RSU. This optimization aims to improve resource utilization and task completion efficiency.

[0064] The base station allocates local computing power according to the DRL algorithm. Within a time slot t, vehicle n can obtain the total number of training rounds expected locally. ,Right now: .

[0065] Where T represents the total duration of a time slot; Tt max Indicates the total time available for computing in a time slot. In real applications, the CPU usually needs to handle multiple tasks at the same time to improve CPU efficiency, such as system maintenance, running other applications, and background processes. These tasks will occupy some CPU resources, resulting in the dispersion of computing power used for training. Therefore, the actual time used for local training is less than T, recorded as T', then the actual total number of local training rounds is It can be expressed as: .

[0066] You can get: .

[0067] .

[0068] Achieving efficient task offloading and resource allocation requires base stations and vehicles to incorporate the long-term impact of their actions in the decision-making process, so this embodiment aims to minimize the sum of the energy consumption of all vehicles in each time slot. Establish the optimization problem: .

[0069] in, .

[0070] .

[0071] .

[0072] .

[0073] p min and p max are the minimum transmission power and the maximum transmission power respectively.

[0074] After the base station allocates the unloading ratio, according to the actual computing capacity of the RSU, the number of unloading tasks assigned to each vehicle may exceed the number of tasks that the vehicle itself needs to handle. Measures the excess of offloaded training rounds.

[0075] Use overload ratio To describe the severity of the overload behavior, that is: .

[0076] As mentioned above, when the uninstall ratio When the value is below a certain threshold q0, vehicle n will not perform the unloading operation. Therefore, the unloading efficiency is defined as an indicator to measure the system's ability to unload tasks, namely: .

[0077] The problem of task offloading and resource allocation can be solved by the DRL algorithm. To this end, DRL is used to find the optimal task offloading and resource allocation scheme. The SAC algorithm was chosen because it is robust and efficient when dealing with continuous action spaces and can effectively balance exploration and utilization. Compared with other DRL algorithms, SAC introduces an entropy term in the objective function to encourage base stations to continuously explore new strategies. This entropy term effectively prevents the base station from converging to a local optimal solution too early during the policy iteration process, thereby avoiding training failure. The goal of the SAC algorithm is to minimize the overall energy consumption of the system, aiming to find an optimal resource allocation scheme.

[0078] Exemplarily, this step includes: Given the local computing power of vehicle n, the actual total number of training iterations is .when No more than , vehicle n can handle all remaining tasks locally, i.e. .when Exceed , it means that vehicle n cannot process all the remaining tasks locally, and the maximum number of iterations that can be processed locally is At the same time, the unprocessed part will be placed in the local buffer. Therefore, the number of iterations of local model training is calculated according to the following formula : .

[0079] The total energy consumption of each vehicle at time slot t is calculated according to the following formula: .

[0080] in, is the total energy consumption of vehicle n in time slot t; is the computing energy consumption of RSU; Calculate energy consumption for local areas; is the transmission energy consumption of RSU; The energy consumed to perform one training iteration for vehicle n; The energy consumed by performing one training iteration for RSU.

[0081] The base station is able to analyze the state and experience in the decision making process. The system state consists of two elements: fading information and vehicle speed Among them, the fading information is determined by the current position of the vehicle, the speed , , remains unchanged in each time slot and changes at the beginning of the next time slot. and Represent the minimum and maximum values ​​of the vehicle speed respectively. Therefore, construct the system state s t expression: .

[0082] in, ; is the fading information of vehicle n at time slot t; is the vehicle speed; ; is the speed of vehicle n at time slot t; N is the total number of vehicles participating in the federated training.

[0083] After obtaining the state, the base station completes the task unloading and power allocation decision for the current time slot. The action is set as three discrete variables: the power when the vehicle sends data to the RSU , CPU calculation frequency assigned by the base station to the vehicle and task offloading ratio Therefore, the action a in time slot t is constructed t expression: .

[0084] in, The power of all vehicles when sending data to RSU; ; is the transmission power of vehicle n transmitting data to RSU; The CPU frequency allocated to all vehicles by the base station; ; The CPU frequency assigned to vehicle n by the base station; The mission unloading ratio for all vehicles; ; is the task offloading ratio of vehicle n.

[0085] Build Rewards t expression: .

[0086] Among them, θ1 is The weight of θ2 is the overload The weight of The weight of .

[0087] S7, update the SAC network according to the system status of the target vehicle, the action and reward of the target time slot.

[0088] The SAC algorithm, or soft actor-critic algorithm, is a reinforcement learning method based on the maximum entropy principle. It contains five networks: 1 actor , 2 critics and , and 2 targets and .

[0089] The core idea of ​​the SAC algorithm is to balance the relationship between exploration and exploitation by increasing the randomness of the strategy. Traditional reinforcement learning algorithms are often prone to falling into local optimality. Even if they perform well in the initial stage, they may not be able to find the global optimal solution when facing a complex environment. SAC encourages the diversity of strategies, allowing the agent to try more possible paths during training and improve the global search ability of the strategy. The objective function of SAC can be expressed as the weighted sum of expected reward and entropy, that is: .

[0090] in, is in state Take action The expected reward, hereinafter referred to as r t . is the policy π in state The entropy under , represents the discount factor. is a temperature parameter that adjusts the randomness of the optimal policy, used to balance the importance of expected reward and entropy.

[0091] This approach allows the agent to maintain a certain degree of randomness in its actions while pursuing high rewards, so that it can better explore the environment and learn strategies. , the corresponding strategy π is recorded as At this time, use Can be adjusted , get the status The optimal solution is denoted as ,Right now: .

[0092] The base station obtains the initial state from the system , and then according to the actor network The strategy selection action . When performing an action After, status Transition to a new state and receive corresponding rewards These interaction data are in the form of tuples The replay buffer R allows random sampling of these tuples, thus breaking the correlation between samples during training. When the number of tuples stored in the replay buffer R reaches the batch size , start updating the five networks of the SAC algorithm.

[0093] Randomly extract M tuples from the replay buffer R, called a mini-batch. For each tuple in the mini-batch ,in ,Will Input to the actor network According to the actor network Strategy , get a new action It is worth noting that With the tuple Different. Then we can calculate The loss function is as follows: .

[0094] Indicates action The dimension of .

[0095] Next, update the actor network : Will Input into two review networks respectively and In the above example, we get the corresponding action value function and The smaller of these two values ​​is defined as Therefore, through the Stochastic Gradient Descent (SGD) algorithm, the actor network The update process is as follows: .

[0096] Where o is the noise sampled from a multivariate normal distribution; is a function for reparameterizing actions Finally, the action network is updated using stochastic gradient descent.

[0097] Next, update the two review networks and : To update the review network, we first transform the mini-batch M Input to two review networks and In the example, we obtain action-value pairs and At the same time, Input to the actor network In the strategy Next action In order to maintain the policy during training In order to improve the randomness and explore a wider range of states and actions, the SAC algorithm introduces an entropy regularization term in the policy function, namely Then, Input to two target networks and In the example, we obtain the target action value pairs and . Take the smaller of these two values ​​and record it as . Then, the target action value can be calculated as follows : .

[0098] Then, update the network through the SGD algorithm: .

[0099] .

[0100] based on , , and , using Adam optimizer, each The actor and comment network are updated once in each time slot. After the actor and comment network are updated, the next time slot is entered. time slots, the two target networks also need to be updated as follows and Parameters: .

[0101] .

[0102] in, Much less than 1; and They are the two updated target networks respectively.

[0103] S8, determining the global model of the base station according to the second local models and the third local models corresponding to all vehicles participating in the federated training.

[0104] After receiving the local models of all vehicles, the base station aggregates the received models. For example, the global model of the base station is constructed expression: .

[0105] in, represents the second local model; represents the third local model; N is the total number of vehicles participating in the federated training.

[0106] S9, repeating the operations of S1-S8 until the preset training rounds of federated training are reached, and the final SAC network and the global model are obtained, so as to allocate resources according to the final SAC network and offload tasks according to the final global model.

[0107] In this embodiment, the values ​​or value ranges of some parameters are as follows: Fig.15 shown.

[0108] like Figure 3 The figure shows the comparison of energy consumption of different methods with five vehicles involved in the training process. The training methods include four DRL methods: SAC, DDPG (Deep Deterministic Policy Gradient), TD3 (Twin Delayed Deep Deterministic Policy Gradient), and PPO (Proximal Policy Optimization), as well as a method that always transmits at maximum power. In terms of energy consumption, the PPO algorithm failed to effectively reduce energy consumption, and its energy consumption value continued to be J. In contrast, the energy consumption of SAC, DDPG and TD3 algorithms shows a rapid downward trend and gradually converges to a relatively stable value. Specifically, the SAC algorithm converges the fastest and is more stable than the other two algorithms. The final energy consumption result is stable at J. This is mainly attributed to the fact that the SAC algorithm balances exploration and utilization during the strategy update process, thus improving the convergence efficiency. Figure 4 As shown, the details on the round interval [1,500] are shown in detail. Figure 4 It can be clearly seen that the fluctuation of TD3 is smaller than that of DDPG, but the DDPG algorithm can explore a lower energy consumption value. Finally, the energy consumption result of the DDPG algorithm stabilizes at J, while the energy consumption of TD3 algorithm is stable at J. This shows that the TD3 algorithm is better at handling noise and uncertainty, making it more stable in the early stages and reaching lower energy consumption faster.

[0109] like Figure 5 Figure 1 shows the comparison of the number of calculations for different methods with five vehicles involved in the training process. The training methods include four DRL methods: SAC, DDPG, TD3, and PPO, as well as a method that always transmits at maximum power. In terms of the number of calculations, all four algorithms eventually converge to a stable value. The goal is to perform as much local training as possible in each round. The results show that the number of calculations for the SAC algorithm gradually and steadily increases and finally stabilizes at times, significantly higher than the other two algorithms. The final calculation times of DDPG and TD3 algorithms are similar, respectively Second and times, but the fluctuation of DDPG algorithm is larger. PPO algorithm has the largest number of calculations and is stable at In theory, a global model trained with more calculations has a higher accuracy.

[0110] like Figure 6 Figure 2 shows a comparison of the reward functions of different methods with five vehicles involved in the training process. Figure 6 It can be seen that the PPO algorithm has the lowest reward value, and the final reward value is the smallest among the four algorithms, stabilizing at This shows that it is not a wise choice. Figure 6 It can also be seen that the four algorithms can eventually converge to a relatively stable reward value. Figure 7 As shown in the figure, the details of the round interval [1,500] are shown in detail. It can be clearly seen that the reward value of the SAC algorithm converges quickly and remains stable after 76 rounds. The DDPG and TD3 algorithms can also reach a relatively stable state after 150 rounds. Figure 8 As shown in the figure, the details of the round interval [2700,3000] are shown. It can be seen that the reward values ​​of the SAC, DDPG and TD3 algorithms finally stabilize at , and .from Figure 8 It can also be seen that the reward values ​​of DDPG and TD3 algorithms tend to surpass the SAC algorithm, but in terms of stability, the SAC algorithm is still the best.

[0111] like Fig. 9 As shown in Figure 2, the number of unloading times under different deep reinforcement learning (DRL) algorithms is shown. Fig. 9 It can be seen that the number of unloading times of DDPG, TD3 and PPO algorithms fluctuates, while the number of unloading times of SAC algorithm shows a trend of steady decline and final convergence.

[0112] like Fig.10As shown, the overload ratio under different DRL algorithms is shown. Except for the PPO algorithm, the overload rates of the other three algorithms all show a downward trend, especially the overload rate of the SAC algorithm, which approaches 0 after the 1332th round. Fig. 9 and Fig.10 It can be seen that the SAC algorithm gradually reduced the number of unloading times during the training process, and trained more locally in the vehicle, thereby reducing the overload rate, and finally achieved a balance between local vehicle training and unloading to the RSU for training. Although the DDPG and TD3 algorithms did not significantly reduce the number of unloading times during the training process, they were still able to reduce the overload rate, which shows that these two algorithms tend to offload training tasks to the RSU for processing and explore an appropriate unloading ratio. Figure 3 This also explains why the energy consumption of these two algorithms is higher than that of the SAC algorithm. The number of unloading times of the PPO algorithm remains high among the four algorithms, but its overload rate is low and remains stable. This shows that the PPO algorithm also tends to unload data to the RSU for processing, and the number of unloading times is close to the number of processing rounds expected by each vehicle locally. The PPO algorithm tends to unload tasks to the RSU for processing, but the number of tasks unloaded each time will not exceed the total number of local tasks by a large margin.

[0113] like Fig.11 As shown, it shows that when setting the threshold and no threshold set From the overall data, we can see that all values ​​are above 0, which indicates that setting the threshold The offloading efficiency is always better than that without setting a threshold This phenomenon verifies the threshold we proposed The method can significantly improve the offloading efficiency. Specifically, among all the reinforcement learning algorithms tested, the TD3 algorithm performed best in improving the offloading efficiency, followed by the SAC and DDPG algorithms, while the PPO algorithm had the smallest improvement effect. The superior performance of the TD3 algorithm may be due to its dual-delayed deep deterministic policy gradient mechanism, which effectively reduces the variance when the policy network is updated, thereby improving the accuracy of the offloading decision. The SAC algorithm also improves the offloading efficiency to a certain extent by introducing the concept of entropy and encouraging exploration of diversity. In contrast, although the PPO algorithm has certain stability and security, its policy updates are relatively conservative, which limits its performance in complex environments.

[0114] like Fig.12 As shown in , the energy consumption of the SAC algorithm under different numbers of vehicles is compared. Fig.13 As shown in Figure 2, the reward functions of the SAC algorithm under different numbers of vehicles are compared. Fig.12 and Fig.13 It can be observed that the energy consumption curve shows a downward trend and gradually stabilizes, while the reward function shows an upward trend and gradually stabilizes. In addition, as the number of vehicles increases, the number of rounds required to achieve a stable state also increases. This shows that in multi-vehicle scenarios, task offloading and resource allocation become more complicated, resulting in more rounds required to achieve convergence to a stable state. Overall, as the number of vehicles involved in training gradually increases, the total energy consumption of the system also increases, while the reward value decreases.

[0115] like Fig.14 As shown in Figure 1, four rounds of testing were conducted, each of which consisted of 50 independent experiments. The results of each round of testing were obtained by calculating the mean of the results of these 50 experiments. Finally, the mean of these four rounds of testing was further averaged to obtain the average of the comprehensive experimental results, and the final experimental results are shown in Fig.14 In. Fig.14 In the box plot, three algorithms with better performance were selected: SAC, DDPG and TD3 algorithms, and the results of these three algorithms were analyzed from three aspects: energy consumption, number of calculations and reward function. In these three aspects, the test results of the model trained by the SAC algorithm are less dispersed, indicating that the data set is concentrated and the fluctuation range is small. In contrast, the test results of the model trained by the DDPG algorithm are relatively dispersed and have a certain degree of variability. Specifically, in Fig.14 In (a), although an outlier appears in the SAC algorithm, the outlier is close to the lower whisker, and the overall data values ​​are concentrated in a small range. The overall distribution is relatively compact and there are fewer outliers. Therefore, the impact of the outlier on the overall analysis can be ignored. Fig.14 In (b), none of the three algorithms have outliers, and it can be seen that the SAC algorithm has the most calculations, which will lead to a higher classification accuracy in federated self-supervised learning. Fig.14 In (c), both DDPG and TD3 algorithms have outliers, especially the outliers of DDPG algorithm are far away from the lower whisker, which further illustrates the relative instability of the two algorithms. In summary, the model trained by SAC algorithm shows high stability and excellent performance.

[0116] In summary, the task offloading and resource allocation method based on federated self-supervised learning provided in this embodiment uses the SAC algorithm to allocate the transmission power from the vehicle to the base station and RSU, the CPU frequency in the vehicle, and the computing resource allocation ratio in the RSU. According to the allocated ratio, the computing resources available to each vehicle in the RSU are calculated. Then, according to the allowed computing resources, each local model training iteration task is divided into local training and RSU-assisted training. An offloading ratio threshold is set to optimize the task offloading efficiency. After offloading part of the iterative tasks to the RSU, the vehicle processes the remaining iterative tasks locally. If there are still unfinished iterations in a round of federated learning, the remaining number of iterations will be stored and wait for the next training, which reduces the system energy consumption and improves the offloading efficiency and accuracy of federated self-supervised learning.

[0117] Example 2.

[0118] Based on the same inventive concept as Example 1, this example provides a task offloading and resource allocation system based on federated self-supervised learning. Since the principle of solving the problem by this system is similar to the task offloading and resource allocation method based on federated self-supervised learning provided in the aforementioned Example 1, the implementation of this system can refer to the implementation of the task offloading and resource allocation method based on federated self-supervised learning provided in Example 1.

[0119] like Fig.16 As shown in FIG. 1 , the task offloading and resource allocation system based on federated self-supervised learning includes: The acquisition module 10 is used to acquire the global model of the federated training stored in the base station and the first transmission power, CPU frequency and task offloading ratio allocated by the base station to the target vehicle.

[0120] The construction module 20 is used to construct a first local model of the target vehicle according to each layer parameter and model structure of the global model.

[0121] The first determination module 30 is used to determine the number of task offloading iterations and the number of untrained iterations according to the CPU frequency and the task offloading ratio.

[0122] The second determination module 40 is used to determine a new local model of the target vehicle according to the first local model and the local data to serve as a second local model.

[0123] The third determination module 50 is used to offload the local data to the RSU at the first transmission power according to the number of task offloading iterations, so as to determine the local model of the RSU as the third local model.

[0124] The fourth determination module 60 is used to upload the second local model, the third local model, the location information of the target vehicle, the speed of the target vehicle in the target time slot, the total energy consumption of the target vehicle and the number of untrained iterations to the base station to determine the system state of the target vehicle, the action and reward of the target time slot.

[0125] The network update module 70 is used to update the SAC network according to the system status of the target vehicle, the action and reward of the target time slot.

[0126] The fifth determination module 80 is used to determine the global model of the base station according to the second local models and the third local models corresponding to all vehicles participating in the federated training.

[0127] The loop module 90 is used to repeatedly execute the operations of the acquisition module to the fifth determination module until the preset training rounds of the federated training are reached, so as to obtain the final SAC network and the global model, so as to allocate resources according to the final SAC network and unload tasks according to the final global model.

[0128] Exemplarily, the first determining module includes: The acquisition unit is used to obtain the total number of training iterations required for vehicle n in time slot t .

[0129] The first determination unit is used to determine the estimated number of training iterations of the local model And the actual number of training iterations of the local model ; The second determination unit is used to determine the total number of training iterations N on the RSU side. R .

[0130] The first calculation unit is used to calculate the expected number of task offloading training iterations on the RSU side according to the following formula : .

[0131] in, is a binary variable, indicating whether vehicle n needs to offload part of its tasks to the RSU; when the task offloading ratio of vehicle n When it is less than the uninstall ratio threshold, ; When the task unloading ratio of vehicle n When the unloading ratio is not less than the threshold, .

[0132] The third determining unit is used to and The minimum value among them is taken as the number of task offloading training iterations .

[0133] The second calculation unit is used to calculate the number of untrained iterations according to the following formula : .

[0134] Exemplarily, the second determining module includes: The third calculation unit is used to calculate the information noise contrast estimation loss according to the following formula : .

[0135] Among them, exp(·) represents an exponential function with a natural constant as the base; For enhanced local data The anchor point sample is input to the function χ which consists of an encoder, two fully connected neural networks and a ReLU function; For enhanced local data The positive sample output after input into the function χ; For enhanced local data The negative sample output after being input to the function χ; τ is the hyperparameter of temperature; K is the total number of local data marked as negative samples; i≠j; π1(·) is the first data enhancement function; π2(·) is the second data enhancement function.

[0136] The fourth calculation unit is used to calculate the loss of dual temperature according to the following formula : .

[0137] Among them, sg[·] represents the stop gradient; The information-noise contrast estimation loss when the temperature hyperparameter is τ2; Denotes the information-noise contrastive estimation loss when the temperature hyperparameter is τ1.

[0138] The first building unit is used to build a second local model according to the loss of the dual temperature expression: .

[0139] in, represents the first local model; η r is the learning rate; is the gradient descent algorithm; Z is the total number of local data.

[0140] Exemplarily, the fourth determining module includes: The third determination unit is used to determine the estimated number of training iterations of the local model And the actual number of training iterations of the local model .

[0141] The fifth calculation unit is used to calculate the number of iterations of local model training according to the following formula : .

[0142] The sixth calculation unit is used to calculate the total energy consumption of each vehicle at time slot t according to the following formula: .

[0143] in, is the total energy consumption of vehicle n in time slot t; is the computing energy consumption of RSU; Calculate energy consumption for local areas; is the transmission energy consumption of RSU; The energy consumed to perform one training iteration for vehicle n; The energy consumed by performing one training iteration for RSU.

[0144] The second building unit is used to build the system state s t expression: .

[0145] in, is the fading information, which is determined by the current position of the vehicle; ; is the fading information of vehicle n at time slot t; is the vehicle speed; ; is the speed of vehicle n at time slot t; N is the total number of vehicles participating in the federated training.

[0146] The third construction unit is used to construct an action a in time slot t t expression: .

[0147] in, The power of all vehicles when sending data to RSU; ; is the transmission power of vehicle n transmitting data to RSU; The CPU frequency allocated to all vehicles by the base station; ; The CPU frequency assigned to vehicle n by the base station; The mission unloading ratio for all vehicles; ; is the task offloading ratio of vehicle n.

[0148] The fourth building block is used to build reward r texpression: .

[0149] Among them, θ1 is The weight of θ2 is the overload The weight of The weight of .

[0150] Exemplarily, the fifth determining module includes: The fifth construction unit is used to construct a global model of the base station expression: .

[0151] in, represents the second local model; represents the third local model; N is the total number of vehicles participating in the federated training.

[0152] For more specific working processes of the above modules, please refer to the corresponding contents disclosed in Example 1, which will not be repeated here.

[0153] Example 3.

[0154] This embodiment provides a computer device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, the steps of the task offloading and resource allocation method based on federated self-supervised learning described in Example 1 are implemented.

[0155] For more specific details of the above method, please refer to the corresponding contents disclosed in Example 1, which will not be repeated here.

[0156] Example 4.

[0157] This embodiment provides a computer-readable storage medium for storing a computer program; when the computer program is executed by a processor, the steps of the task offloading and resource allocation method based on federated self-supervised learning described in Example 1 are implemented.

[0158] For more specific details of the above method, please refer to the corresponding contents disclosed in Example 1, which will not be repeated here.

[0159] Example 5.

[0160] This embodiment provides a computer program product, including computer executable instructions or a computer program. When the computer executable instructions or the computer program are executed by a processor, the steps of the task offloading and resource allocation method based on federated self-supervised learning described in Example 1 are implemented.

[0161] For more specific details of the above method, please refer to the corresponding contents disclosed in Example 1, which will not be repeated here.

[0162] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the systems, devices, storage media, and computer program products disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0163] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0164] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0165] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0166] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0167] The present invention has been described in detail above in conjunction with specific implementations and exemplary examples, but these descriptions cannot be understood as limiting the present invention. Those skilled in the art understand that, without departing from the spirit and scope of the present invention, a variety of equivalent substitutions, modifications or improvements may be made to the technical solution of the present invention and its implementation methods, all of which fall within the scope of the present invention. The scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A method for task offloading and resource allocation based on federated self-supervised learning, characterized in that: include: S1, obtaining a global model of federated training stored in a base station and a first transmission power, CPU frequency, and task offloading ratio allocated by the base station to a target vehicle; S2, constructing a first local model of the target vehicle according to each layer parameter and model structure of the global model; S3, determining the number of task offloading iterations and the number of untrained iterations according to the CPU frequency and the task offloading ratio; S4, determining a new local model of the target vehicle according to the first local model and the local data to serve as a second local model; S5, unloading the local data to the RSU at the first transmission power according to the number of task offloading iterations, so as to determine a local model of the RSU and use it as a third local model; S6, uploading the second local model, the third local model, the location information of the target vehicle, the speed of the target vehicle in the target time slot, the total energy consumption of the target vehicle, and the number of untrained iterations to the base station to determine the system state of the target vehicle, the action and the reward of the target time slot; S7, updating the SAC network according to the system state of the target vehicle, the action and reward of the target time slot; S8, determining a global model of the base station according to the second local models and the third local models corresponding to all vehicles participating in the federated training; S9, repeating the operations of S1-S8 until the preset training rounds of federated training are reached, and the final SAC network and the global model are obtained, so as to allocate resources according to the final SAC network and offload tasks according to the final global model.

2. The task offloading and resource allocation method according to claim 1, characterized in that: The determining the number of task offloading iterations and the number of untrained iterations according to the CPU frequency and the task offloading ratio includes: Get the total number of training iterations required for vehicle n in time slot t ; Determine the expected number of training iterations for the local model And the actual number of training iterations of the local model ; Determine the total number of training iterations N on the RSU side R ; The expected number of task offloading training iterations on the RSU side is calculated according to the following formula : ； in, is a binary variable, indicating whether vehicle n needs to offload part of its tasks to the RSU; when the task offloading ratio of vehicle n When it is less than the uninstall ratio threshold, ; When the task unloading ratio of vehicle n When the unloading ratio is not less than the threshold, ; Will and The minimum value among them is taken as the number of task offloading training iterations ; The number of untrained iterations is calculated according to the following formula : 。 3. The task offloading and resource allocation method according to claim 1, characterized in that: The step of determining a new local model of the target vehicle according to the first local model and the local data as a second local model includes: The information-noise contrast estimation loss is calculated according to the following formula : ； Among them, exp(·) represents an exponential function with a natural constant as the base; For enhanced local data The anchor point sample is input to the function χ which consists of an encoder, two fully connected neural networks and a ReLU function; For enhanced local data The positive sample output after input into the function χ; For enhanced local data The negative sample output after being input to the function χ; τ is the hyperparameter of temperature; K is the total number of local data marked as negative samples; i≠j; π1(·) is the first data enhancement function; π2(·) is the second data enhancement function; The loss of dual temperature is calculated according to the following formula : ； Among them, sg[·] represents the stop gradient; The information-noise contrast estimation loss when the temperature hyperparameter is τ2; The information-noise contrast estimation loss when the temperature hyperparameter is τ1; Constructing a second local model based on the loss of dual temperature expression: ； in, represents the first local model; η r is the learning rate; is the gradient descent algorithm; Z is the total number of local data.

4. The task offloading and resource allocation method according to claim 2, characterized in that: The second local model, the third local model, the location information of the target vehicle, the speed of the target vehicle in the target time slot, the total energy consumption of the target vehicle and the number of untrained iterations are uploaded to the base station to determine the global model of the base station, the system state of the target vehicle, the action and reward of the target time slot, including: The number of iterations of local model training is calculated according to the following formula : ； The total energy consumption of each vehicle at time slot t is calculated according to the following formula: ； in, is the total energy consumption of vehicle n in time slot t; is the computing energy consumption of RSU; Calculate energy consumption for local areas; is the transmission energy consumption of RSU; The energy consumed to perform one training iteration for vehicle n; The energy consumed by performing one training iteration for the RSU; Build system status t expression: ； in, is the fading information, which is determined by the current position of the vehicle; ; is the fading information of vehicle n at time slot t; is the vehicle speed; ; is the speed of vehicle n at time slot t; N is the total number of vehicles participating in the federated training; Construct action a in time slot t t expression: ； in, The power of all vehicles when sending data to RSU; ; is the transmission power of vehicle n transmitting data to RSU; The CPU frequency allocated to all vehicles by the base station; ; The CPU frequency assigned to vehicle n by the base station; mission unloading ratio for all vehicles; ; is the task unloading ratio of vehicle n; Build Rewards t expression: ； Among them, θ1 is The weight of θ2 is the overload The weight of The weight of .

5. The task offloading and resource allocation method according to claim 1, characterized in that: The determining of the global model of the base station according to the second local model and the third local model corresponding to all vehicles participating in the federated training includes: Build a global model of base stations expression: ； in, represents the second local model; represents the third local model; N is the total number of vehicles participating in the federated training.

6. A task offloading and resource allocation system based on federated self-supervised learning, characterized in that: include: An acquisition module, used to acquire a global model of federated training stored in a base station and a first transmission power, a CPU frequency, and a task offloading ratio allocated by the base station to a target vehicle; A construction module, used for constructing a first local model of the target vehicle according to each layer parameter and model structure of the global model; A first determination module is used to determine the number of task offloading iterations and the number of untrained iterations according to the CPU frequency and the task offloading ratio; A second determination module, used for determining a new local model of the target vehicle according to the first local model and the local data, to serve as a second local model; A third determination module is used to offload the local data to the RSU at a first transmission power according to the number of task offloading iterations, so as to determine a local model of the RSU as a third local model; a fourth determination module, configured to upload the second local model, the third local model, the location information of the target vehicle, the speed of the target vehicle in the target time slot, the total energy consumption of the target vehicle, and the number of untrained iterations to the base station to determine the system state of the target vehicle, the action and the reward in the target time slot; A network update module, used to update the SAC network according to the system status of the target vehicle, the action and reward of the target time slot; A fifth determination module, configured to determine a global model of the base station according to the second local models and the third local models corresponding to all vehicles participating in the federated training; A loop module is used to repeatedly execute the operations of the acquisition module to the fifth determination module until a preset training round of federated training is reached, so as to obtain a final SAC network and a global model, so as to perform resource allocation according to the final SAC network and perform task unloading according to the final global model.

7. The task offloading and resource allocation system according to claim 6, characterized in that: The first determining module comprises: The acquisition unit is used to obtain the total number of training iterations required for vehicle n in time slot t ; The first determination unit is used to determine the estimated number of training iterations of the local model And the actual number of training iterations of the local model ; The second determination unit is used to determine the total number of training iterations N on the RSU side. R ; The first calculation unit is used to calculate the expected number of task offloading training iterations on the RSU side according to the following formula : ； in, It is a binary variable, indicating whether vehicle n needs to offload part of its tasks to the RSU; when the task offloading ratio of vehicle n When it is less than the uninstall ratio threshold, ; When the task unloading ratio of vehicle n When the unloading ratio is not less than the threshold, ; The third determining unit is used to and The minimum value among them is taken as the number of task offloading training iterations ; The second calculation unit is used to calculate the number of untrained iterations according to the following formula : 。 8. The task offloading and resource allocation system according to claim 6, characterized in that: The second determining module comprises: The third calculation unit is used to calculate the information noise contrast estimation loss according to the following formula : ； Among them, exp(·) represents an exponential function with a natural constant as the base; For enhanced local data The anchor point sample is input to the function χ which consists of an encoder, two fully connected neural networks and a ReLU function; For enhanced local data The positive sample output after input into the function χ; For enhanced local data The negative sample output after being input to the function χ; τ is the hyperparameter of temperature; K is the total number of local data marked as negative samples; i≠j; π1(·) is the first data enhancement function; π2(·) is the second data enhancement function; The fourth calculation unit is used to calculate the loss of dual temperature according to the following formula : ； Among them, sg[·] represents the stop gradient; The information-noise contrast estimation loss when the temperature hyperparameter is τ2; The information-noise contrast estimation loss when the temperature hyperparameter is τ1; The first building unit is used to build a second local model according to the loss of the dual temperature expression: ； in, represents the first local model; η r is the learning rate; is the gradient descent algorithm; Z is the total number of local data.

9. The task offloading and resource allocation system according to claim 7, characterized in that: The fourth determination module comprises: The fifth calculation unit is used to calculate the number of iterations of local model training according to the following formula : ； The sixth calculation unit is used to calculate the total energy consumption of each vehicle at time slot t according to the following formula: ； in, is the total energy consumption of vehicle n in time slot t; is the computing energy consumption of RSU; Calculate energy consumption for local areas; is the transmission energy consumption of RSU; The energy consumed to perform one training iteration for vehicle n; The energy consumed by performing one training iteration for the RSU; The second building unit is used to build the system state s t expression: ； in, is the fading information, which is determined by the current position of the vehicle; ; is the fading information of vehicle n at time slot t; is the vehicle speed; ; is the speed of vehicle n at time slot t; N is the total number of vehicles participating in the federated training; The third construction unit is used to construct an action a in time slot t t expression: ； in, The power of all vehicles when sending data to RSU; ; is the transmission power of vehicle n transmitting data to RSU; The CPU frequency allocated to all vehicles by the base station; ; The CPU frequency assigned to vehicle n by the base station; mission unloading ratio for all vehicles; ; is the task unloading ratio of vehicle n; The fourth building block is used to build reward r t expression: ； Among them, θ1 is The weight of θ2 is the overload The weight of The weight of .

10. The task offloading and resource allocation system according to claim 6, characterized in that: The fifth determining module comprises: The fifth construction unit is used to construct a global model of the base station expression: ； in, represents the second local model; represents the third local model; N is the total number of vehicles participating in the federated training.

Images