Energy-saving automatic interconnected vehicle service unloading method based on deep reinforcement learning

Abstract

The invention discloses an energy-saving automatic interconnected vehicle service unloading method based on deep reinforcement learning, and provides a distributed service unloading method named ECAC. The ECAC maps A3C to an end-edge-cloud cooperative system based on an asynchronous deep reinforcement learning method, namely an asynchronous dominant actor-commentator (A3C) algorithm. The core idea of the method is that ECD is mapped into an intelligent agent in A3C, and an unloading decision for vehicle service is guided; and mapping the cloud server into a central network in the A3C for overall planning and summarizing learning results of the intelligent agents and copying the parameters of the cloud server to the corresponding ECD after each time of parameter updating. The whole algorithm has the characteristics of dynamic demand learning of the system and automatic adjustment of an unloading strategy, and can meet the demands of services with different time delay sensitivities. And the problems of energy consumption and time delay can be considered for a long time, and green and high-quality service is realized.

Description

technical field [0001] The invention belongs to the technical field of edge computing, and in particular relates to an energy-saving automatic interconnected vehicle service unloading method based on deep reinforcement learning. Background technique [0002] Connected autonomous vehicles (CAVs) are autonomous vehicles enabled by vehicle-to-everything (V2X) communication, and are the product of the joint development of the Internet of Vehicles and automatic control technologies. On the basis of autonomous vehicles, CAVs connect to other vehicles, roadside units and external servers through wireless communication technology, making them a cooperating whole. In CAVs, vehicle decision-making no longer only depends on the data collection of on-board sensors and the calculation of on-board computing devices, but also on the shared data of other vehicles and road agents, as well as the support of external servers, such as edge computing and cloud computing. A significant advantage...

AI Technical Summary

Problems solved by technology

However, these studies using DRL did not make full use of the distributed characteristics of ECDs themselves in the edge-cloud collaborative mode, but carried out the learning of each ECD offloading scheme in isolation, which not only increased the learning burden of the system, but also It is not conducive to improving the generalization ability of the system to the environment

Finally, there are still some offloading methods that cannot dynamically weigh the impact of delay and energy consumption on the decision-making results according to the type of service, or do not consider energy consumption at all.

This algorithm considers both energy consumption and delay when calculating the cost, but it cannot dynamically change the weight of energy consumption and delay according to the task type

Images