Self-adaptive task unloading method oriented to vehicle-assisted MEC network

Abstract

The invention discloses a self-adaptive task unloading method oriented to a vehicle-assisted MEC network. The method comprises the following steps: establishing a vehicle task unloading scene of vehicle task unloading; next, with minimization of total energy consumption and maximization of task unloading efficiency as optimization objectives, communication and resources in a vehicle task unloading scene are scheduled by using an SFRL learning framework, and a vehicle-assisted MEC transmission model is constructed; task unloading is carried out according to the vehicle auxiliary MEC transmission model; and training and updating the vehicle auxiliary MEC transmission model based on an SFRL optimization algorithm, and finally realizing vehicle task unloading in the vehicle task unloading scene. By combining the advantages of segmentation learning and federal reinforcement learning, the resource utilization efficiency of distributed learning is optimized, the calculation and communication cost is reduced, the data privacy protection level is improved, and the learning efficiency, the energy efficiency and the overall performance of the system are remarkably enhanced.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle networking, and particularly relates to an adaptive task offloading method for a vehicle-assisted MEC network. Background Art

[0002] In recent years, the development of graphics processing units has greatly improved the computing power of terminal devices, making distributed computing more attractive in edge environments. This computing mode allows end-users to perform learning locally and achieve model aggregation on the MEC server by only exchanging model parameters instead of transmitting complete data. Federated Reinforcement Learning (FRL) is a decentralized learning technology that was initially applied in wireless networks. This method allows mobile devices (MDs) to perform learning locally and only send model updates to the MEC server for aggregation, avoiding large-scale data transmission. Specifically, FRL adopts parallel model learning technology, where each MD obtains the global model from the MEC server and synchronously trains it on the local sub-dataset, and then sends the model update back to the MEC server for aggregation. This method has significant advantages in reducing communication requirements, but it depends on the MDs having sufficient computing power, and its communication cost increases with the increase in model complexity.

[0003] In contrast, Split Learning (SL), as a distributed learning method, divides the machine learning model into multiple sub-models through extraction layers for distributed execution between the terminal and the MEC server. SL only deploys a small number of levels of the user-level model (MD model) at the terminal device side, and runs the MEC server-level model at the MEC server side, and data exchange is completed through truncated data. SL is outstanding in reducing the computing burden of MDs, and its communication cost is proportional to the size of the user dataset, especially suitable for MDs with limited resources. However, due to the relatively high communication requirements in each round of SL, as the number of network users increases, the communication cost will further increase, affecting its overall performance.

[0004] In the prior art, the Split Federated Learning (SFL) method combines the advantages of FL and SL. However, similar to SL, SFL still incurs a high communication cost because each user adopts the SL mode, resulting in a communication bottleneck problem. Meanwhile, wireless transmission of learning variables is required in both SFL and SL. However, in an environment where wireless resources such as bandwidth and energy are limited, the transmission process is significantly restricted, and only a small number of MDs can complete the broadcast of learning variables in one round. Therefore, the user scheduling strategy becomes crucial. The limited resources of users pose significant challenges in the implementation of distributed learning algorithms, and there is an urgent need for a solution that can minimize communication and computational energy consumption while meeting the learning accuracy requirements. Summary of the Invention

[0005] To solve the problems existing in the background art, the object of the present invention is to provide an adaptive task offloading method for a vehicle-assisted MEC network. The present invention ensures the optimal operation of the system under different resource conditions by intelligently allocating the SL or FRL method, thereby improving energy efficiency and resource utilization while enhancing learning accuracy.

[0006] The technical solution adopted by the present invention is as follows, including the following steps:

[0007] Step S1: First, establish a vehicle task offloading scenario for vehicle task offloading in a computer.

[0008] Step S2: Then, the processor in the computer uses the SFRL learning framework to schedule communication and resources in the vehicle task offloading scenario with the optimization goals of minimizing the overall energy consumption and maximizing the task offloading efficiency, and constructs a vehicle-assisted MEC transmission model.

[0009] Step S3: The processor performs task offloading according to the vehicle-assisted MEC transmission model.

[0010] Step S4: The processor trains and updates the vehicle-assisted MEC transmission model based on the SFRL optimization algorithm, and finally realizes the vehicle task offloading in the vehicle task offloading scenario.

[0011] The specific content of the said Step S1 is as follows:

[0012] First, introduce mobile edge computing (MEC) into the vehicle task offloading scenario in the computer, and construct the traffic environment of the vehicle with tasks to be offloaded. Then, based on the SFRL learning framework, optimize and model the communication delay and energy consumption in the vehicle task offloading scenario.

[0013] The specific content of the said Step S2 is as follows:

[0014] Optimize the communication and computing resource scheduling in the vehicle task offloading scenario based on the SFRL learning framework. Aiming to minimize the overall energy consumption and maximize the task offloading efficiency, establish a vehicle-assisted MEC transmission model using the SFRL learning framework to achieve the collaborative optimization of communication and computing resources.

[0015] In the step S1, for vehicle u in the vehicle task offloading scenario n The communication delay Is obtained according to the following formula:

[0016]

[0017] Where Represents the total delay when the task of the nth user u n Is offloaded to the mobile device vehicle u i ; Represents the communication delay between the working module at the time of offloading in vehicle u n And vehicle u i ; Represents the computing delay of vehicle u n at the time of offloading; W n Represents the size of the working module in vehicle u n ; Represents the CPU frequency within vehicle u n ; Represents the CPU frequency of vehicle u i ; α represents a given delay coefficient; r DBL (u n , u i ) represents the transmission rate between vehicle u n and vehicle u i .

[0018] In the step S1, for vehicle u in the vehicle task offloading scenario n The total energy consumption Is obtained according to the following formula:

[0019]

[0020] Where Represents the total energy consumption when the task of user u n is offloaded to the mobile device vehicle u i ; Represents the communication energy consumption between the working module at the time of offloading in vehicle u n and vehicle u i ; Represents the computing energy consumption of vehicle u n at the time of offloading; Represents the computing power of vehicle u n ; uN Denote the Nth vehicle as u N , where N represents the total number of vehicles; P trans denotes the transmission power, and P rec denotes the received power; Z represents the number of working modules.

[0021] The vehicle-assisted MEC transmission model in step S2 is constructed based on the SFRL learning framework, and the transmission delay and energy consumption of the SFRL learning framework are obtained through the following formulas:

[0022]

[0023] where T represents the transmission delay; denotes the scheduling variable of vehicle u when the FRL technology is adopted n ; denotes the local model update time of vehicle u when the FRL technology is adopted n ; denotes the power of vehicle u when the FRL technology is adopted n ; denotes the energy consumed by the local model of vehicle u when the FRL technology is adopted n ; denotes the scheduling parameter of vehicle u when the SL technology is adopted n ; denotes the local model update time of vehicle u when the SL technology is adopted n ; denotes the power of vehicle u when the SL technology is adopted n ; denotes the local model update time of vehicle u when the SL technology is adopted n ; denotes the energy consumed by the local model of vehicle u when the SL technology is adopted n ; denotes the local model update time of vehicle u when the FRL technology is adopted n ; denotes the transmission time of vehicle u when the FRL technology is adopted n ; denotes the local model update time of vehicle u when the SL technology is adopted n ; denotes the uplink transmission time of vehicle u when the SL technology is adopted n ; denotes the downlink transmission time of vehicle u when the SL technology is adopted n ;

[0024] In step S2, the optimization objective of the vehicle-assisted MEC transmission model is as follows:

[0025]

[0026] The constraint conditions of the vehicle-assisted MEC transmission model are as follows:

[0027]

[0028] T ≤ Τ 0

[0029]

[0030] Wherein, represents a binary decision variable, indicating whether the task of user u n is offloaded to the mobile device vehicle u i ; I represents the set of all users selected to participate in the learning process; represents the energy consumption when the task of user u n is offloaded to the mobile device vehicle u i ; represents the set of mobile devices that can establish an offloading relationship with user u n ; W represents the parameters of the vehicle-assisted MEC transmission model; L(W) represents the global loss function; represents the delay when the task of user u n is offloaded to the mobile device vehicle u i ; T 0 represents the maximum delay limit for each round; represents the bandwidth ratio allocated to user u n ;

[0031] In the step S4, the specific method for training and updating the vehicle-assisted MEC transmission model based on the SFRL optimization algorithm is as follows:

[0032] Obtain the gradient of the global loss function in the vehicle-assisted MEC transmission model, where the gradients of the global loss function are combined for forward propagation, and at the same time, backward propagation is also performed to update the parameters of the vehicle-assisted MEC transmission model, realizing the training of the vehicle-assisted MEC transmission model.

[0033] A vehicle networking content caching and transmission optimization system:

[0034] Includes a vehicle task offloading scenario construction module for establishing a vehicle task offloading scenario for vehicle tasks;

[0035] Includes a vehicle-assisted MEC transmission model construction model for scheduling communication and resources in the vehicle task offloading scenario with the optimization goal of minimizing the overall energy consumption and maximizing the task offloading efficiency, and constructing a vehicle-assisted MEC transmission model;

[0036] It includes a task offloading module for task offloading according to the vehicle-assisted MEC transmission model;

[0037] It includes a vehicle adaptive task offloading module: for training and updating the vehicle-assisted MEC transmission model based on the SFRL optimization algorithm, and finally realizing vehicle task offloading in the vehicle task offloading scenario.

[0038] The key innovations of the present invention include the following aspects:

[0039] 1. Energy-saving adaptive distributed learning framework: The present invention proposes a SFRL (Split Federated Reinforcement Learning) framework, which combines the advantages of split learning (SL) and federated reinforcement learning (FRL) to intelligently select suitable learning methods to achieve the optimal energy efficiency and resource utilization rate.

[0040] 2. Dynamic allocation learning method: According to the computing resources, bandwidth conditions and data distribution characteristics of the terminal devices, the FRL or SL method is flexibly allocated to each device, thereby optimizing the overall performance of the system. This dynamic adaptive scheduling mechanism improves the learning efficiency in diverse vehicle-assisted MEC networks.

[0041] 3. Markov decision process (MDP) modeling of task offloading: The task offloading problem is redefined as an MDP problem, providing an intelligent decision-making framework to optimize resource allocation and energy usage.

[0042] 4. Energy-optimized user scheduling strategy: A user scheduling strategy based on energy optimization is designed to improve the accuracy and communication efficiency of model training by selecting appropriate mobile device (MD) groups and learning methods in a bandwidth and resource-constrained environment.

[0043] 5. Efficient model parameter exchange: Under the condition of limited network bandwidth, this framework can reduce communication overhead and improve model training efficiency by selecting suitable learning modes and effective parameter exchange.

[0044] 6. Co-optimization of communication and computing costs: Under the conditions of limited bandwidth and resources, by optimizing the user scheduling strategy and learning method selection, the communication and computing overhead of the system are reduced, enabling the advantages of SL and FRL to be fully exerted, especially suitable for diverse data distributions and complex network conditions.

[0045] 7. Privacy protection of distributed learning models: The present invention realizes efficient model parameter exchange and optimization on the premise of ensuring data privacy, and solves the deficiencies of existing methods in privacy protection and efficient data collaborative sharing.

[0046] 8. Adapting to diverse vehicle-assisted MEC networks: This framework can flexibly adjust the learning mode in the case of strong data heterogeneity and resource heterogeneity, and effectively cope with scenarios with limited bandwidth and computing power.

[0047] The purpose of the present invention is to propose an energy-saving adaptive distributed learning framework, SFRL, which fully combines the advantages of FRL and SL and optimizes system energy efficiency and resource utilization by intelligently selecting learning methods. In the SFRL framework of the present invention, FRL or SL methods are flexibly assigned to each device according to the computing resources, bandwidth conditions and data distribution characteristics of the terminal device, so as to achieve energy-efficient task offloading. To this end, the present invention redefines the task offloading problem as a Markov decision process (MDP) problem, and designs an energy-optimized user scheduling strategy to maximize the accuracy of model training and communication efficiency by selecting appropriate MDs groups and learning methods under bandwidth and resource constraints.

[0048] The present invention effectively solves the problems of high communication cost, heavy computing burden and poor resource adaptability in the prior art, and provides an efficient and energy-saving solution for large-scale AI model task offloading in vehicle-assisted MEC networks.

[0049] The beneficial effects of the present invention are:

[0050] 1. The present invention combines the advantages of segmented learning and federated reinforcement learning, selects the learning mode in an intelligent and adaptive manner, optimizes the resource utilization efficiency of distributed learning, reduces computing and communication costs, improves the level of data privacy protection, and can flexibly adapt to the diverse application requirements in the vehicle-assisted MEC network, significantly enhancing the learning efficiency, energy efficiency and overall performance of the system.

[0051] 2. Considering computing resources and bandwidth limitations: Existing federated reinforcement learning (FRL) relies on the terminal device to have high computing power to support local model training, but this puts a burden on devices with limited resources; while segmented learning (SL) reduces the computing burden of terminal devices, its communication cost increases with the size of the data set and causes delays when updating sequentially. The present invention intelligently selects the appropriate learning method (SL or FRL) so that the system can flexibly allocate tasks according to the computing resources and bandwidth of the device, thereby effectively reducing the dual burden of computing and communication. Therefore, the present invention reduces resource consumption and communication delays while ensuring system performance.

[0052] 3. Optimize communication and energy efficiency: In a bandwidth-constrained environment, traditional methods cannot achieve efficient distributed learning due to high communication costs. The SFRL framework proposed in this invention models the task offloading problem as a Markov Decision Process (MDP). On the premise of ensuring learning accuracy, it optimally selects scheduling strategies and intelligently allocates learning methods, enabling each device to obtain optimal energy and bandwidth allocation under different resource conditions, and significantly optimizing the communication and energy efficiency of the system.

[0053] 4. Enhance data privacy protection: This invention makes full use of the decentralized characteristics of federated learning to achieve efficient exchange of model parameters on the premise of ensuring data privacy, avoiding the transmission of large-scale data, and effectively solving the problems in privacy protection and data collaborative sharing. At the same time, by dynamically selecting FRL or SL, flexible privacy protection strategies can also be implemented according to task requirements.

[0054] 5. Adapt to complex and diverse network scenarios: The vehicle-assisted MEC network has high heterogeneity, complex data distribution, and diverse user requirements. Traditional methods cannot effectively handle problems such as bandwidth limitations, data heterogeneity, and resource heterogeneity. The adaptive distributed learning framework SFRL of this invention can flexibly adapt to different application scenarios by dynamically allocating SL or FRL, enabling it to maintain high-efficiency and stable learning performance even when resource conditions change.

[0055] 6. Improve learning efficiency and model accuracy: By intelligently selecting SL and FRL in the MEC network, this invention improves the model training efficiency while ensuring high model accuracy. Especially in complex environments, through optimized user scheduling strategies and task offloading methods, this invention can effectively allocate bandwidth and computing resources, reduce learning instability caused by device performance differences, and thus enhance the accuracy and consistency of the distributed learning model.

[0056] 7. Energy saving and scalability: The design of this invention enables the system to be applicable in MEC networks of different scales. And due to the adoption of energy-optimized scheduling strategies, it can significantly reduce energy consumption in large-scale scenarios, ensuring the energy-saving and scalability of the system. Brief Description of the Drawings

[0057] Figure 1 Schematic diagram of the application of the SFRL method in a vehicle-assisted MEC network;

[0058] Figure 2 Flowchart of the method of this invention. Detailed Description of the Invention

[0059] The present invention will be described in detail below in conjunction with specific implementation cases. The following implementation cases will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form.

[0060] The embodiments of the present invention include the following steps, as Figure 2 shown:

[0061] Step S1: First, establish a vehicle task offloading scenario for vehicle task offloading in a computer;

[0062] Step S2: Then, with the goal of minimizing the overall energy consumption and maximizing the task offloading efficiency, the processor in the computer uses the SFRL learning framework to schedule the communication and resources in the vehicle task offloading scenario and constructs a vehicle-assisted MEC transmission model;

[0063] Step S3: The processor performs task offloading according to the vehicle-assisted MEC transmission model;

[0064] Step S4: The processor trains and updates the vehicle-assisted MEC transmission model based on the SFRL optimization algorithm. The updated vehicle-assisted MEC transmission model will be sent back to each mobile device vehicle, and finally, the vehicle task offloading in the vehicle task offloading scenario is realized.

[0065] The present invention establishes a vehicle-assisted MEC transmission model, in which a group of vehicles Nc = {u 1 , u 2 ,... u N} cooperate with an MEC server to train a machine learning model for data analysis and inference. As Figure 1 shown, it is expected that each user (i.e., vehicle) u i has a local dataset D i , and the data volume of the local dataset D i is denoted as |D i |. In each round, only some mobile devices (MDs) are selected to participate in the transmission model learning. Each user can be scheduled through the federated reinforcement learning (FRL) or split learning (SL) method. A group consisting of mobile devices and a group of vehicles is selected, and a specific user (vehicle) u n is excluded due to weak channel characteristics or energy constraints. The channel fading model between the user and the mobile edge computing MEC server is established by evaluating the spatial expected value of the path loss of the line-of-sight LoS and non-line-of-sight NLoS populations. The main goal of the MEC server is to obtain a general model by utilizing the communication resources of vehicle-assisted MEC, and the ultimate goal is to minimize the global loss function L(W). The data volume corresponding to a specific vehicle u n is denoted as the local loss function It can be expressed in the corresponding form:

[0066]

[0067] where y j represents the actual value of the data point ; represents the predicted value of the global model W at the data point , that is, the output of the global model W for this data point .

[0068] The central server is responsible for collecting and aggregating the local model (LM) updates from FRL and SL. As the FRL aggregator, the global model (GM) is updated by the central server in each round while the SL model is being learned. At the beginning of the process, the central server performs forward propagation and synchronously learns the data of SL. The resulting truncated data gradients are then fed back to the sending vehicles for backpropagation. Subsequently, the server updates the global model (GM) using the FedAvg algorithm by calculating the weighted average of the gradients generated by the truncated data of each vehicle during the backpropagation process. Finally, the central server executes the FedAvg algorithm on the gradients of the local updates and transmits the FedAvg algorithm back to all vehicles. In short, the SFRL mechanism cleverly combines the advantages of FL and SL, thus reducing the learning pressure and communication overhead. Each vehicle can choose to execute the module locally according to its available computing power or offload it to adjacent vehicles. Therefore, the overall latency of the module depends on this decision, whether to execute locally or offload.

[0069] In specific implementation, step S1 is specifically as follows:

[0070] First, introduce mobile edge computing MEC into the vehicle task offloading scenario in the computer and construct the traffic environment of the vehicle with tasks to be offloaded. Then, based on the SFRL learning framework, optimize the communication delay and energy consumption in the vehicle task offloading scenario.

[0071] Among them, step S2 is specifically as follows:

[0072] Optimize the communication and computing resource scheduling in the vehicle task offloading scenario based on the SFRL learning framework (i.e., the adaptive distributed learning framework). With the goal of minimizing the overall energy consumption and maximizing the task offloading efficiency, establish a vehicle-assisted MEC transmission model using the SFRL learning framework to achieve the collaborative optimization of communication and computing resources.

[0073] In step S2, based on the SFRL learning framework, first select a suitable subset of MDs to ensure that these subsets of MDs have sufficient learning ability and can participate in the training of the model. The selection of the subset of MDs should be based on their characteristics such as energy consumption, computing power, communication latency, etc. For each MD, determine whether to use the FRL method or the SL method through the SFRL learning framework.

[0074] Step S3 is specifically as follows:

[0075] Under the SFRL learning framework of the vehicle task offloading scenario, the task offloading is completed by training the subset of MDs and the MEC server. Each MD learns based on its own local data and then collaboratively learns with the MEC server through the SFRL learning framework.

[0076] Step S4 is specifically as follows: Use the SFRL algorithm to optimize the task offloading strategy through the following process: First, each mobile device vehicle MD uses local data for learning and updates the local model (SL). Then, send the locally updated encrypted model parameters to the central server for federated averaging (FRL). Finally, the updated model will be sent back to each mobile device vehicle MD for local fine-tuning. Finally, the vehicle task offloading in the vehicle network is realized.

[0077] In step S1, vehicle u in the vehicle task offloading scenario n The communication latency is processed and obtained according to the following formula:

[0078]

[0079] where represents the total latency when the task of the nth user u n is offloaded to the mobile device vehicle u i ; represents the communication latency between the vehicle u n and the vehicle u i when the working module is offloaded, and the working module is loaded in each vehicle; represents the computing latency of the vehicle u n when offloading; W n represents the size of the working module in the vehicle u n ; represents the CPU frequency within the vehicle u n ; represents the CPU frequency of the vehicle u i ; α represents a given latency coefficient; r DBL (u n ,u i ) represents the vehicle u n and the vehicle u iThe transmission rate between.

[0080] In a specific implementation, each vehicle u n can choose to execute the work module locally according to its available computing power, or offload the work module to an adjacent vehicle. Therefore, the overall latency of the work module depends on this decision, that is, whether to execute locally or offload. If vehicle u n executes the work module locally, then the local latency of vehicle u n is In the case of task offloading, there are two different latency components, namely communication latency and computing latency. The communication latency component is related to the transmission of the work module between vehicles, while the computing latency component is due to the possible unavailability of the processor CPU of vehicle u n during the execution of the work module, resulting in computing latency. In the offloading case, a certain degree of latency is generated for the specific work module deployed on each vehicle. In this case, the complete offloading of the work module is performed, rather than only offloading a part. In the offloading case, the computing latency of vehicle u n is In the offloading case, the communication latency of vehicle u n is Among them, represents the communication latency generated when vehicle u n offloads the work module to the adjacent vehicle u i and generates the communication latency.

[0081] In step S1, in the vehicle task offloading scenario, the overall energy consumption of vehicle u n is obtained by processing according to the following formula:

[0082]

[0083] Among them, represents the total energy consumption when the task of user u n is offloaded to the mobile device vehicle u i ; represents the communication energy consumption between vehicle u n and vehicle u i during offloading; represents the computing energy consumption of vehicle u n during offloading; represents the computing power of vehicle u n ; u N represents the Nth vehicle u N , N represents the total number of vehicles; P trans represents the transmission power, P rec represents the receiving power; Z represents the number of work modules.

[0084] Analyze the energy consumption model related to the computing, communication, and movement of vehicles. For each vehicle u n Regarding the resource capacity, the working module can be executed locally or transferred to an adjacent vehicle for execution. It should be noted that energy is consumed during the transmission of the working module, and a certain amount of energy is also consumed during the execution of the working module. Therefore, the total energy consumption generated during the offloading process of the working module can be determined by considering the energy consumption related to communication, computing, and movement. In vehicle operations, the computing energy consumption when each vehicle executes locally can be represented by to represent. The energy consumption related to computing during the offloading process can be expressed as Derive the communication energy consumption through the computing process based on the transmission of the working module between vehicles. The total communication energy consumption can be determined by the following formula:

[0085]

[0086] The total energy consumption can finally be determined, and the total energy consumption is composed of communication energy consumption and computing energy consumption together.

[0087] The vehicle-assisted MEC transmission model in step S2 is constructed based on the SFRL learning framework. The transmission delay and energy consumption in a single round of the SFRL learning framework are obtained according to the following formulas:

[0088]

[0089] Among them, T represents the transmission delay; represents the scheduling variable of vehicle u n when using the FRL technology; represents the local model update time of vehicle u n when using the FRL technology; represents the power of vehicle u n when using the FRL technology; represents the energy consumed by the local model of vehicle u n when using the FRL technology; represents the scheduling parameter of vehicle u n when using the SL technology; represents the local model update time of vehicle u n when using the SL technology; represents the power of vehicle u n when using the SL technology; represents the local model update time of vehicle u n when using the SL technology; represents the energy consumed by the local model of vehicle u n when using the SL technology; represents the local model update time of vehicle u n when using the FRL technology; Vehicle u when using the FRL technology n Transmission time; Vehicle u when using the SL technology n Local model update time; Vehicle u when using the SL technology n Uplink transmission time; Vehicle u when using the SL technology n Downlink transmission time;

[0090] Among them, Indicates vehicle u n Is scheduled to use the FRL method for scheduling, Indicates vehicle u n Is not scheduled to use the FRL method for scheduling, while Indicates vehicle u n Uses the SL method for scheduling, Indicates vehicle u n Does not use the SL method for scheduling, if Holds, it means that vehicle u n Is not selected to participate in this round;

[0091] In the above formula, for vehicle u n The transmission time is obtained by processing according to the following formula:

[0092]

[0093] Among them, Represents the communication consumption of vehicle u n ; Represents vehicle u n The uplink transmission rate to the MEC server; Represents vehicle u n The size of the user-level MD model of; Represents vehicle u n The size of the activation value of the MD model of; r s Represents vehicle u n The downlink data rate of.

[0094] The vehicle-assisted MEC transmission model is constructed based on the energy-saving adaptive distributed learning framework SFRL. When implementing the SFRL framework, in the local model LM learning stage, each target user uses its local dataset and the received global model GM to calculate the local model LM update through the FRL or SL method. The FRL method allows the mobile devices MDs on the vehicle to independently schedule the calculation of the local model update of the vehicle. Let Represents the computing power of the mobile devices MDs on the vehicle, measured in CPU cycles per second; while Denote user u n Calculate the number of CPU cycles required for a single sample data. Thus, user u can be determined n Calculate the computing time required for its local model update

[0095]

[0096] Among them, Denote that when using the FRL method, vehicle u n Calculate vehicle u n The computing time required for local model update; Denote the number of iterations for vehicle u n To perform local learning; Denote that when using the FRL method, vehicle u n The number of CPU cycles required to obtain a single sample data; D n Denote vehicle u n The corresponding local dataset; K F Denote the set of all mobile devices using the FRL calculation method.

[0097] The individuals scheduled by the SL method calculate the update of their local model LM by collaborating with the MEC server. During this process, each user calculates the LM update of the received MD model, while the MEC server is responsible for calculating the LM update of its own MEC server model. Therefore, user u n The number of CPU cycles required to calculate a single sample data is In addition Denote the number of CPU cycles required for the base station to calculate a single sample data. It can be observed that Therefore, user u participating in the LM update calculation of the MD model and the MEC server model n Can determine:

[0098]

[0099]

[0100] Among them, Denote that when using the SL method, vehicle u n Calculate vehicle u n The computing time required for local model update; The computing time required for the base station to calculate the local model update of vehicle u using the SL method n ; Denote that when using the SL method, vehicle u n The number of CPU cycles required to obtain a single sample data; Denote the number of CPU cycles required for the base station to obtain a single sample data when using the SL method; KS Denote the set of all mobile devices adopting the SL calculation method.

[0101] Assume that all clients use frequency-division multiple access (FDMA) to transmit their model variables to the MEC server. User u n The uplink transmission rate to the MEC server Can be determined by the following expression:

[0102]

[0103] Where, Denotes the bandwidth ratio of user u n That is, the allocation ratio of this user in the total bandwidth; B w Denotes the total available bandwidth of the system; Denotes user u n Channel gain; Denotes user u n Transmission power when uploading data; N 0 Denotes the given power spectral density.

[0104] When adopting the FRL technology, the user only needs to upload the updated local model LM to the MEC server. The communication consumption depends on the size of the model, denoted by . Therefore, the transmission time Can be expressed as:

[0105]

[0106] In the SL method, the mobile devices (MDs) need to transmit the output activation values of the slice layer and the updated MD model to the MEC server. The communication cost generated by this process can be divided into two parts: the size of the MD model And the size of the activation values Where, |D i | Denotes the size of the local dataset, a denotes the given activation value coefficient; the uplink transmission time Can be expressed as:

[0107]

[0108] In addition, the downlink broadcast of the MD model is ignored, and the downlink transmission only includes the output gradient of the slice layer of the MEC server model, and the gradient size is g denotes the given gradient coefficient; assume that the MEC server uses all the bandwidth when sending the gradient to each MD, then the downlink data rate r s The expression is as follows:

[0109]

[0110] Among them, B s represents the total bandwidth of the MEC server; p s represents the transmission power of the MEC server; N s represents the noise power spectral density of the MEC server. Therefore, the downlink transmission time can be defined as:

[0111]

[0112] In step S2, the optimization objective of the vehicle-assisted MEC transmission model is as follows:

[0113]

[0114] The constraint conditions of the vehicle-assisted MEC transmission model are as follows:

[0115]

[0116] T ≤ Τ 0

[0117]

[0118] Among them, represents a binary decision variable, indicating whether the task of user u n is offloaded to the mobile device vehicle u i ; I represents the set of all users selected to participate in the learning process; represents the energy consumption when the task of user u n is offloaded to the mobile device vehicle u i ; represents the set of mobile devices that can establish an offloading relationship with user u n ; W represents the parameters of the vehicle-assisted MEC transmission model; L(W) represents the global loss function; represents the delay when the task of user u n is offloaded to the mobile device vehicle u i ; T 0 represents the maximum delay limit for each round; represents the bandwidth ratio allocated to user u n .

[0119] In step S4, the specific method for training and updating the vehicle-assisted MEC transmission model based on the SFRL optimization algorithm is as follows:

[0120] Obtain the gradient of the global loss function in the vehicle-assisted MEC transmission model, where the gradients of the global loss function are combined for forward propagation, and at the same time, backpropagation is also performed to update the parameters of the vehicle-assisted MEC transmission model, realizing the training of the vehicle-assisted MEC transmission model.

[0121] A vehicle - Internet - of - Things content caching and transmission optimization system:

[0122] It includes a vehicle task offloading scenario construction module, which is used to establish a vehicle task offloading scenario for vehicle task offloading;

[0123] It includes a vehicle - assisted MEC transmission model construction model, which takes minimizing the total energy consumption and maximizing the task offloading efficiency as the optimization goal, uses the SFRL learning framework to schedule communication and resources in the vehicle task offloading scenario, and constructs a vehicle - assisted MEC transmission model;

[0124] It includes a task offloading module, which is used to perform task offloading according to the vehicle - assisted MEC transmission model;

[0125] It includes a vehicle adaptive task offloading module: which is used to train and update the vehicle - assisted MEC transmission model based on the SFRL optimization algorithm, and finally realize the vehicle task offloading in the vehicle task offloading scenario.

[0126] Problem formulation: To address the challenges brought by wireless channel random fading and energy - constrained MDs, the present invention explores an MDs scheduling scheme focusing on energy efficiency for implementing FRL in wireless networks. For this purpose, it is necessary to introduce the SFRL framework into various wireless networks with different data distributions and computing capabilities. This requires redesigning an energy - efficient user scheduling method that can select a representative subset of MDs and determine their model learning methods in each round.

[0127] The present invention formulates the above - mentioned problem as an optimization problem, whose objective is to minimize the total energy consumption of MDs and maximize user diversity while meeting the delay constraint. The present invention considers the energy consumption of MDs, including local communication and computing energy consumption. It is worth noting that a wider range of MDs is preferentially selected in each round. In addition, each MD has the ability to choose the FRL or SL method, which is represented as a set. The formula of the optimization problem is as follows:

[0128]

[0129] T≤Τ 0

[0130]

[0131] Where, represents a binary decision variable, indicating whether the task of user u n is offloaded to the mobile device vehicle u i If it means that user u n is offloaded to the mobile device vehicle u iotherwise Let \(I\) denote the set of all users selected to participate in the learning process; Let \(u\) denote the vehicle when using the FRL technology n Scheduling variable; Let \(u\) denote the user n whose task is offloaded to the mobile device vehicle \(u\) i The energy consumption of vehicle \(u\), including the energy consumption of local computing and communication; \(W\) represents the parameter of the vehicle-assisted MEC transmission model, and the goal is to minimize the loss of the model to improve the learning accuracy; \(L(W)\) represents the global loss function, which is used to measure the performance of the machine learning model; Let \(u\) denote the user n whose task is offloaded to the mobile device vehicle \(u\) i The latency when the task of user \(u\) is offloaded to the mobile device vehicle \(u\); \(T\) 0 denotes the maximum latency limit for each round; Let \(u\) denote the user n The allocated bandwidth ratio.

[0132] On the premise of satisfying the constraint conditions C1 to C5, when: takes the minimum value, and \(L(W)\) takes the smallest value as much as possible, it is the optimal solution of the vehicle-assisted MEC transmission model.

[0133] To solve this problem, the present invention first reformulates the above problem as a standard multiple-choice knapsack problem (such as problem P2), and ignores the time constraint C1 during the preliminary solution. The algorithm designed by the present invention is used to solve P2 to obtain a preliminary feasible solution. Subsequently, based on this feasible solution, further search for the final optimal solution that satisfies the constraint C1. For the scheduling problem of mobile devices (MDs), the present invention effectively equates it to a multiple-choice knapsack problem, and selects an appropriate learning method for each user to minimize the total energy consumption of MDs, maximize the diversity among MDs, and ensure that the total bandwidth (BW) is within the available bandwidth range.

[0134] The present invention proposes an innovative split federated reinforcement learning algorithm, which combines the advantages of FRL and SL. First, the present invention transforms the joint optimization problem into a Markov decision process (MDP) and constructs its learning mechanism. Finally, the present invention designs a novel SFRL algorithm to improve the energy efficiency and performance of the vehicle-assisted MEC network. In the present invention, it is assumed that an agent is deployed in each vehicle-assisted MEC network. The agent collects and characterizes key environmental information through SFRL to generate the state \(S\). The following is a more detailed MDP model. The agent generates an action \(A\) according to the state \(S\), and after execution, the environment returns a reward \(R\) and updates to the next state. Thus, the present invention represents the basic elements of the MDP model as a triple \(\{S, A, R\}\), and trains the agent based on this data to gradually optimize the decision-making.

[0135] State: In time slot \(t\), the configuration of the vehicle-assisted MEC transmission model can be divided into three parts: the device itself, the channel, and the edge server. When making a decision, the agent needs to obtain the data of the vehicle-assisted MEC network. Therefore, the state \(S(t)\) includes the current energy information and the queue length of the vehicle-assisted MEC network. In addition, the transmission rate of the vehicle-assisted MEC network is determined by the channel state and other vehicle-assisted MEC networks in the same channel. Considering the optimization objective, it is necessary to determine the queue length of the edge server as part of the state \(S(t)\). \(S(t)\) can be expressed as:

[0136] \(S(t)=\{E\) u (t),L u (t),cg u,i ,L i (t)\}

[0137] where \(S(t)\) represents the state containing the current environmental information of the system; \(E\) u (t) represents the current energy state of the vehicle device; \(L\) u (t) represents the queue state of the vehicle device in time slot \(t\), referring to the amount of data or tasks to be processed; \(cg\) u,i represents the channel gain of the vehicle-assisted MEC network device under different channels; \(L\) i (t) represents the current task or data processing queue length of the edge server;

[0138] Action: During the time slot \(t\), the agent must choose between energy acquisition and task offloading. At the same time, when the vehicle-assisted MEC transmission model selects offloading, the agent also needs to regulate the transmission capacity and select an edge server for offloading. Therefore, the action of the present invention can be expressed as:

[0139]

[0140] b u,i (t)\(\in\{0,1\}\)

[0141] where \(A\) u (t) represents the decision or operation taken by the vehicle-assisted MEC network in time slot \(t\); \(b\) u,i (t) represents the edge vehicle selected for task offloading. \(b\) u,i (t)=1 means that user \(u\) n offloads the task to vehicle \(i\), and \(b\) u,i (t)=0 means no offloading; represents the power allocated by the vehicle during task offloading to support data transmission, and \(n\) u,x (t) represents the offloading method, where \(n\) u,F (t)=1 means vehicle \(u\)n is scheduled to use the FRL method, while n u,S (t) = 1 indicates that the SL method is used for model learning.

[0142] Reward: The goal of the agent is to maximize the reward, so it is crucial to construct a suitable reward function. In the method of the present invention, the reward function needs to consider energy usage, optimization, and edge server load balancing. In addition, the load balancing of the edge server requires the cooperation of the vehicle-assisted MEC network. The reward function also needs to consider the impact of the selected edge server on the vehicle-assisted MEC network and the overall goal. Based on the above criteria, the following reward function is designed:

[0143] R(t) = {αp1 + βp2}

[0144] R u (t) represents the reward of the agent at time slot t, which is used to evaluate the effect of the action taken by the agent; p1 represents the total energy consumption of the system after taking action A, that is, the value of the objective function problem p1; p2 represents the global loss function of the system after taking action A, that is, the value of the objective function problem p2; α, β represent weight coefficients, which are used to adjust the relative importance of each factor in the reward function.

[0145] The scalability of the MDP model plays a key role in the application effect of SFRL. As the complexity and scale of the MDP increase, the computational burden will also increase accordingly, which may limit the scalability of the SFRL method. Large-scale MDPs may require a large amount of computational resources and time to complete the learning and decision-making processes. Therefore, the scalability of the MDP model directly affects the efficiency and feasibility of SFRL implementation. To address the scalability challenge, researchers can explore techniques such as approximation methods, parallel computing, or distributed learning frameworks. Ensuring scalability enables SFRL to handle increasingly complex environments and tasks, thus achieving more robust and adaptable learning performance in diverse practical applications.

[0146] Learning method based on SFRL: The present invention proposes an innovative SFRL algorithm that covers the detailed learning mechanism in the system model. This SFRL algorithm allows a part of the MDs to select the SL method for implementation, while allowing another part of the MDs to use the FRL method. In the SFRL algorithm proposed in the present invention, a specific group of MDs (denoted as ) cooperate with the MEC server to use their local datasets to train a complete FRL model, similar to the SL method. The FRL model is divided into two sub-models, namely the MDs-side model and the MEC server model. In this framework, the MDs receive the MDs model and train it, while the MEC server trains the MEC server model. All SL MDs (denoted as u n , where )Train the MDs model in parallel using its local dataset until the cut layer. Subsequently, the behavior values obtained from the cut layer are transmitted to the MEC server. As a high-resource entity, the MEC server quickly completes model learning by performing forward propagation on the MEC server model using the behavior values from the SL MDs to calculate the loss function where

[0147] Next, the gradient of the loss function is comprehensively calculated and backpropagation is performed at the cut layer. At this layer, the gradient is accurately calculated and returned to the SL MDs so that they can perform backpropagation and local update of the MDs model respectively. Therefore, for user u in the SL MDs n the local model update can be expressed as:

[0148]

[0149] where represents the model weight update amount of the SL MDs (vehicle-assisted MEC transmission model) at time t; n t represents the learning rate at time t; represents the gradient calculated by the SL MDs at time t; N S represents the number of mobile devices MDs using the supervised learning (SL) method; N F represents the number of mobile devices MDs using the federated learning (FL) method; represents the model weight of the SL MDs at time t;

[0150] Sequential learning can be achieved on the MEC server using SL. Therefore, the SL device u belonging to the second type of MD n (where n ∈ {1, 2,..., N S}) will obtain more LM updates This means that if, in a certain communication cycle, the number of federated learning (FL) devices and SL devices is equal, i.e., N F = N S , then SL will provide more LM updates.

[0151] SFRL-based Task Offloading Method: The SFRL algorithm proposed in this invention is expected to be applied to vehicle-assisted MEC networks, benefiting from the energy resources, heterogeneous computing capabilities, and different data distributions of MDs. The SFRL algorithm combines the advantages of SL and FRL. MDs with lower computing capabilities can benefit from the SL method and only learn part of the levels of the ML model locally, while high-performance MDs with large datasets can utilize federated reinforcement learning to train the entire ML model locally.

[0152] This invention aims to solve the problem of efficient distributed learning in the process of the integration of mobile edge computing (MEC) networks and the Internet of Vehicles (IoV), especially the problem of achieving high-precision model training and optimizing wireless communication performance under diverse data distributions and complex network conditions. First, the existing technologies face the problem of being unable to efficiently implement model training in diverse data distributions and complex network environments, especially how to achieve effective data collaboration and sharing under the premise of privacy protection. Second, in a dynamically changing network environment, traditional methods are difficult to achieve efficient resource allocation, resulting in the stability and continuity of model training being affected. In addition, when existing distributed learning algorithms are applied in vehicle-assisted MEC networks, there is a lack of a mechanism that can adapt to different data characteristics and flexibly switch between split learning (SL) and federated reinforcement learning (FRL), making it difficult to achieve the optimal energy consumption and latency performance. This invention innovatively proposes a hybrid learning framework that combines SL and FRL, which can not only achieve efficient model parameter exchange when the network bandwidth is limited but also intelligently select the appropriate learning mode according to data characteristics, thereby optimizing the overall performance. This innovative solution has significant advantages in improving the efficiency of distributed learning, protecting data privacy, and enhancing the reliability of wireless communication, which are the main innovation points and technical features of this invention.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive task offloading method for vehicle-assisted MEC networks, characterized in that: The following steps are involved: Step S1, first, establishing a vehicle task unloading scenario for vehicle task unloading in a computer; Step S2: Next, the processor in the computer uses the SFRL learning framework to schedule the communication and resources in the vehicle task offloading scenario and build a vehicle-assisted MEC transmission model with the optimization goal of minimizing overall energy consumption and maximizing task offloading efficiency; Step S3: The processor performs task offloading according to the vehicle-assisted MEC transmission model; Step S4: The processor updates the vehicle-assisted MEC transmission model based on SFRL optimization algorithm training, and finally realizes vehicle task offloading in the vehicle task offloading scenario.

2. The method for adaptive task offloading in vehicle-assisted MEC network according to claim 1, characterized in that: The step S1 is specifically as follows: Firstly, mobile edge computing (MEC) is introduced into the vehicle task offloading scenario in the computer, and the traffic environment of the vehicle to be offloaded is constructed. Then, based on the SFRL learning framework, the communication delay and energy consumption in the vehicle task offloading scenario are optimized and modeled.

3. The method for adaptive task offloading in vehicle-assisted MEC network according to claim 1, characterized in that: The step S2 is specifically as follows: Based on the SFRL learning framework, the communication and computing resource scheduling in the vehicle task offloading scenario is optimized, with the goal of minimizing the overall energy consumption and maximizing the task offloading efficiency. The vehicle-assisted MEC transmission model is established using the SFRL learning framework to achieve the coordinated optimization of communication and computing resources.

4. The method for adaptive task offloading in vehicle-assisted MEC network according to claim 1, characterized in that: In step S1, vehicle u in the vehicle task unloading scenario n Communication delay According to the following formula: in, represents the nth user u n The task is offloaded to the mobile device vehicle u i Total delay of Indicates that the working module is in vehicle u when unloading n and vehicle u i Communication delay between Indicates vehicle u when unloading n The calculation delay of W n Represents vehicle u n The size of the working module; Represents vehicle u n CPU frequency inside; Represents vehicle u i CPU frequency; α represents a given delay coefficient; r DBL (u n ,u i ) represents vehicle u n and vehicle u i The transmission rate between.

5. The method for adaptive task offloading in vehicle-assisted MEC network according to claim 1, characterized in that: In step S1, vehicle u in the vehicle task unloading scenario n Overall energy consumption According to the following formula: in, Represents user u n The task is offloaded to the mobile device vehicle u i Total energy consumption; Indicates that the working module is in vehicle u when unloading n and vehicle u i Communication energy consumption between Indicates vehicle u when unloading n The computing energy consumption of Represents vehicle u n The computing power of u N represents the Nth vehicle u N , N represents the total number of vehicles; P trans Represents the transmission power, P rec represents the received power; Z represents the number of working modules.

6. The method for adaptive task offloading in vehicle-assisted MEC network according to claim 1, characterized in that: The vehicle-assisted MEC transmission model in step S2 is constructed based on the SFRL learning framework. The transmission delay and energy consumption of the SFRL learning framework are obtained according to the following formula: Where T represents the transmission delay; β unF Indicates the vehicle u when FRL technology is used n Scheduling variables; Indicates the vehicle u when FRL technology is used n Local model update time; Indicates the vehicle u when FRL technology is used n Power; Indicates the vehicle u when FRL technology is used n The energy consumed by the local model; Indicates the vehicle u when using SL technology n Scheduling parameters; Indicates the vehicle u when using SL technology n Local model update time; Indicates the vehicle u when using SL technology n Power; Indicates the vehicle u when using SL technology n Local model update time; Indicates the vehicle u when using SL technology n The energy consumed by the local model; Indicates the vehicle u when FRL technology is used n Local model update time; Indicates the vehicle u when FRL technology is used n Transmission time; Indicates the vehicle u when using SL technology n Local model update time; Indicates the vehicle u when using SL technology n Uplink transmission time; Indicates the vehicle u when using SL technology n Downlink transmission time.

7. The method for adaptive task offloading in vehicle-assisted MEC network according to claim 1, characterized in that: In step S2, the optimization objectives of the vehicle-assisted MEC transmission model are as follows: The constraints of the vehicle-assisted MEC transmission model are as follows: T≤Τ0 in, represents a binary decision variable, representing user u n Whether the task is offloaded to the mobile device vehicle u i Above; I represents the set of all users selected to participate in the learning process; Represents user u n The task is offloaded to the mobile device vehicle u i Energy consumption; Indicates that it can communicate with user u n The set of mobile devices that establish an offloading relationship; W represents the parameters of the vehicle-assisted MEC transmission model; L(W) represents the global loss function; Represents user u n The task is offloaded to the mobile device vehicle u i T0 represents the maximum delay limit of each round; Represents user u n The allocated bandwidth ratio.

8. The method for adaptive task offloading in vehicle-assisted MEC network according to claim 1, characterized in that: In step S4, the specific method of training and updating the vehicle-assisted MEC transmission model based on the SFRL optimization algorithm is as follows: The gradient of the global loss function in the vehicle-assisted MEC transmission model is obtained, wherein the gradient of the global loss function is combined together for forward propagation, and backpropagation is also performed to update the parameters of the vehicle-assisted MEC transmission model to realize the training of the vehicle-assisted MEC transmission model.

9. A vehicle networking content caching and transmission optimization system for implementing any of the methods described in claims 1-8, characterized in that: It includes a vehicle task unloading scenario building module for building a vehicle task unloading scenario for vehicle task unloading; It includes a vehicle-assisted MEC transmission model construction model, which is used to schedule communications and resources in vehicle task offloading scenarios using the SFRL learning framework with the optimization goals of minimizing overall energy consumption and maximizing task offloading efficiency, and construct a vehicle-assisted MEC transmission model; including a task offloading module for performing task offloading according to a vehicle-assisted MEC transmission model; It includes a vehicle adaptive task offloading module: used to train and update the vehicle-assisted MEC transmission model based on the SFRL optimization algorithm, and finally realize the vehicle task offloading in the vehicle task offloading scenario.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the adaptive task offloading method according to any one of claims 1 to 8 are implemented.

Images