Power distribution network communication method for electric vehicle and 5G base station access

Abstract

The invention discloses a power distribution network communication method for electric vehicle and 5G base station access, and the method comprises the steps: S1, generating a task set based on collected related data of a charging pile and a 5G base station; s2, constructing a task unloading scheduling model by taking the minimum total cost for executing all tasks in the task set as a target; s3, the task unloading scheduling model is solved through a Q-Learning algorithm, and an optimal task unloading scheduling strategy is obtained; s4, performing unloading scheduling on the tasks in the task set according to the optimal task unloading scheduling strategy, and unloading the tasks to servers of corresponding levels for execution; s5, executing the task through the server of the corresponding level, and predicting future power load distribution; s6, future power load distribution is provided for the power distribution network, and the power distribution network is assisted to make load management and scheduling decisions. According to the method, the power distribution network can be effectively assisted to carry out more accurate and efficient load management and scheduling decisions.

Description

Technical Field

[0001] The present invention relates to the technical field of distribution network control, and in particular to a distribution network communication method for electric vehicles and 5G base station access. Background Art

[0002] With the popularization of electric vehicles and the rapid development of 5G technology, more and more electric vehicles and 5G base stations are connected to the distribution network, which has brought unprecedented challenges to the control methods of traditional distribution networks. Among them, electric vehicles and 5G base stations are new loads, and their access methods, power consumption characteristics and demands are significantly different from those of traditional loads, making traditional distribution network control methods unable to cope with these new changes.

[0003] The charging behavior of electric vehicles is random and decentralized, which increases the difficulty of predicting the load of the power grid. Electric vehicle users usually decide the time and place of charging according to their own travel needs and the availability of charging facilities, resulting in extremely uneven distribution of charging load in time and space. This uneven distribution aggravates the peak-to-valley difference of the power grid load, increases the demand for peak and frequency regulation of the power grid, reduces the load rate of the power grid, increases the demand for backup capacity of the power grid, and significantly changes the operation mode of the large power grid. At the same time, the traditional distribution network planning method is mainly based on historical load data and electricity consumption characteristics for prediction and planning, while the access of electric vehicles changes the load structure and characteristics of the distribution network, making the traditional planning criteria no longer applicable. The access of a large number of electric vehicles requires the addition of charging facilities. The construction and operation of these facilities will have a huge impact on the distribution capacity setting, distribution line selection, relay protection setting, etc. of the distribution network. In addition, the charging behavior of electric vehicles also has a certain degree of randomness and uncertainty, which further increases the difficulty of load prediction and planning in distribution network planning.

[0004] The access of 5G base stations to the distribution network also poses challenges to the traditional distribution network control methods. As a new type of communication infrastructure, the power consumption characteristics and demands of 5G base stations are significantly different from those of traditional loads. 5G base stations usually use DC power supply and need to be equipped with energy storage equipment and uninterruptible power supplies to ensure the reliability and stability of power supply. The large-scale access of these devices increases the complexity and difficulty of control of the distribution network. At the same time, the power load of 5G base stations is highly time-sensitive and volatile, which requires the distribution network to respond quickly and flexibly to meet its power demand. Traditional methods are mainly based on historical data and experience for control and decision-making, and it is difficult to adapt to the new changes and uncertainties brought about by the access of electric vehicles and 5G base stations.

[0005] In summary, the access of electric vehicles and 5G base stations to the distribution network has brought unprecedented challenges to the traditional distribution network control methods. Therefore, new control methods and technologies need to be developed to meet these challenges. For example, technologies such as big data, artificial intelligence, and the Internet of Things can be used to predict and analyze the power consumption behavior of electric vehicles and 5G base stations to achieve more accurate and flexible control. However, when a large number of prediction and analysis tasks appear, how to offload and schedule each task (i.e., offload the task to the corresponding level of server execution) has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] In view of the deficiencies of the above-mentioned prior art, the technical problem to be solved by the present invention is: how to provide a distribution network communication method for electric vehicles and 5G base station access, construct a task offloading scheduling model with the goal of minimizing the total cost, and solve the task offloading scheduling model through an improved Q-Learning algorithm to obtain the optimal task offloading scheduling strategy, so that the task can be offloaded to the server of the corresponding level for execution while considering the costs of multiple aspects such as computing resources, communication resources and energy consumption, thereby effectively assisting the distribution network to make more accurate and efficient load management and scheduling decisions.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] A distribution network communication method for electric vehicles and 5G base station access, comprising:

[0009] S1: Generate a task set based on the collected data of charging piles and 5G base stations;

[0010] S2: Construct a task offloading scheduling model with the goal of minimizing the total cost of executing all tasks in the task set;

[0011] S3: Solve the task offloading scheduling model through the Q-Learning algorithm to obtain the optimal task offloading scheduling strategy;

[0012] S4: Unload and schedule the tasks in the task set according to the optimal task unloading scheduling strategy, and unload the tasks to the corresponding level of server for execution;

[0013] S5: predict the future power load distribution by executing tasks through servers at the corresponding level;

[0014] S6: Provide the future power load distribution to the distribution network to assist the distribution network in making decisions on load management and scheduling.

[0015] Preferably, in step S1, the task is to predict the future power load distribution based on the collected relevant data of charging piles and 5G base stations.

[0016] Preferably, in step S5, a load forecasting model is constructed and trained based on a long short-term memory network; the trained load forecasting model is deployed on a server; the collected relevant data of charging piles and 5G base stations are input into the trained load forecasting model deployed on the server, and the corresponding future power load distribution is output.

[0017] Preferably, in step S3, the optimal task offloading scheduling strategy is used to offload the tasks in the task set to servers at the terminal device layer, edge computing service layer or cloud computing layer for execution.

[0018] Preferably, in step S3, the objective function of the task offloading scheduling model is expressed as:

[0019]

[0020] Where: G i represents the objective function; Z ik (Δ) represents task T ik The total cost; Δ = 0 means that the task is offloaded to the server of the terminal device layer, Δ = 1 means that the task is offloaded to the server of the edge computing layer, Δ = 2 means that the task is offloaded to the server of the cloud computing layer, and Δ = -1 means that the task transmission fails; Represents task T ik The total cost of server execution at the end device level; Represents task T ik The total cost of executing on edge computing layer servers; Represents task T ik The total cost of server execution at the cloud computing layer.

[0021] Preferably, in step S3, the constraints of the task offloading scheduling model include:

[0022] 1) The tasks assigned to the server cannot exceed its maximum computing capacity:

[0023]

[0024] Where: Respectively represent the execution server s i The kth task T generated ik The number of instructions required, the limit time, and the start time; represents the maximum computing capacity of the server; K represents the number of tasks in the task set;

[0025] 2) The total delay of executing the task cannot exceed the time limit:

[0026]

[0027] Where: Indicates the execution of task T ik The total delay.

[0028] Preferably, in step S3, the task T is calculated by the following steps: ik Total cost of server execution at the end device level

[0029] 1) Computation task T ik Delay caused by server execution at the terminal device layer

[0030]

[0031] Where: Indicates the execution of task T ik The number of instructions required; Indicates the maximum computing capacity of the terminal device layer server;

[0032] 2) Computation task T ik Energy consumption generated by server execution at the terminal device layer

[0033]

[0034] Where: Indicates the energy consumption of each instruction processed by the terminal device layer server;

[0035] 3) Computation task T ik Total cost of server execution at the end device level

[0036]

[0037] Where: α represents the weight of energy consumption.

[0038] Preferably, in step S3, the task T is calculated by the following steps: ik Total cost of server execution at the edge computing layer

[0039] 1) Computation task T ik The latency caused by server execution at the edge computing layer

[0040]

[0041] Where: Indicates the execution of task T ik The number of instructions required; Indicates the maximum computing capacity of the edge computing layer server; Indicates the execution of task T ik Required amount of input and output data; Lie Indicates server i Maximum transmission rate to edge computing layer servers; represents the bandwidth of the edge computing layer server; S represents the useful signal power; σ 2 is the noise power;

[0042] 2) Computation task T ik Energy consumption generated by server execution at the edge computing layer

[0043]

[0044] Where: Indicates the energy consumption of each instruction transmitted by the edge computing layer server; Indicates the energy consumption of each instruction processed by the edge computing layer server;

[0045] 3) Computation task T ik Total cost of server execution at the edge computing layer

[0046]

[0047] Where: α represents the weight of energy consumption.

[0048] Preferably, in step S3, the task T is calculated by the following steps: ik Total cost of server execution at the cloud computing layer

[0049] 1) Computation task T ik The latency caused by server execution in the cloud computing layer

[0050]

[0051] Where: Indicates the execution of task T ik The number of instructions required; Indicates the maximum computing capacity of the cloud computing layer server; Represents task T ik Required amount of input and output data; L ic For servers i Maximum transfer rate to cloud computing layer servers; represents the bandwidth of the cloud computing layer server; S represents the useful signal power; σ 2 is the noise power;

[0052] 2) Computation task T ik Energy consumption generated by server execution at the cloud computing layer

[0053]

[0054] Where: It represents the energy consumption of each instruction transmitted by the cloud computing layer server; It represents the energy consumption of each instruction processed by the cloud computing layer server;

[0055] 3) Computation task T ik Total cost of server execution at the cloud computing layer

[0056]

[0057] Where: α represents the weight of energy consumption.

[0058] Preferably, in step S4, the Q-Learning algorithm is improved; the processing steps of the improved Q-Learning algorithm are as follows:

[0059] S401: construct a Q table to store the Q value of each action taken in each state;

[0060] S402: Setting the learning rate β, discount factor γ, exploration rate ε, penalty factor ζ, decay coefficient ω, value coefficient η of the algorithm; initial environment state;

[0061] S403: For each task t i ∈Task Set T i , perform the following steps:

[0062] S404: Obtain the current environment state CS; determine the currently available action set A according to the environment state CS; select action a from the action set A based on the ε-greedy strategy; wherein action a is executed at the terminal device layer, offloaded to the edge computing layer, offloaded to the cloud computing layer, or querying the cloud computing layer server upward;

[0063] S405: Execute action a and obtain a new environment state CS′ after the execution of action a;

[0064] S406: Calculate the reward according to the environment state CS and the environment state CS′ in combination with the objective function of the task offloading scheduling model;

[0065] 1) When the action is executed at the terminal device layer, offloaded to the edge computing layer, or offloaded to the cloud computing layer, the reward is calculated using the following formula:

[0066]

[0067] Where: Z i,k Indicates the execution of task T ik Rewards; Indicates the execution of task T ik The delay generated Indicates the execution of task T ik The energy consumption generated, where the superscript d = l, e, c, respectively, represents the server of the terminal device layer, edge computing service layer or cloud computing layer; Represents task T ik time limit;

[0068] 2) When the action is to query the cloud computing layer server upward, the reward is calculated by the following formula:

[0069] Z i,k ″=Z i,k ′·ω;

[0070] Z i,k ′=Z i,k η;

[0071] Where: Z i,k ″ indicates the reward for upward query; Z i,k Indicates the execution of task T ik The reward; η represents the value coefficient; ω represents the attenuation coefficient, where each upward query requires an increase in the attenuation coefficient ω;

[0072] S407: Calculate the current Q value according to action a, environment state CS′ and reward Z, and update the Q table;

[0073] The calculation formula of Q value is:

[0074]

[0075] Where: a t 、cs t , Z t Represent the action, environment state and reward at time t respectively;

[0076] S408: Repeat steps S403 to S407 until all tasks are completed and the final Q table is output;

[0077] S409: Selecting the most valuable offloading action for each task based on the Q table to form an optimal task offloading scheduling strategy.

[0078] Compared with the prior art, the distribution network communication method for electric vehicles and 5G base station access in the present invention has the following beneficial effects:

[0079] In the present invention, firstly, relevant data of charging piles and 5G base stations (such as charging demand, base station energy consumption, user behavior patterns, etc.) are collected, and a prediction task set is generated through the edge device layer and scheduled through the algorithm, thereby ensuring the effectiveness and pertinence of the task set. Secondly, by constructing a task offloading scheduling model with the goal of minimizing the total cost of executing all tasks in the task set, the costs of computing resources, communication resources, and energy consumption can be considered; at the same time, by solving the task offloading scheduling model with an improved Q-Learning algorithm, it is possible to dynamically adapt to changes in the load of the distribution network, electric vehicles, and 5G base stations and find the optimal task offloading scheduling strategy, so that tasks can be offloaded to the corresponding level of servers for execution while considering the costs of computing resources, communication resources, and energy consumption. This can not only reduce the operating cost of the distribution network, but also improve the overall efficiency and flexibility of the distribution network. Then, according to the optimal task offloading scheduling strategy, the tasks can be automatically and intelligently offloaded to the corresponding level of server execution, ensuring efficient task processing and reasonable resource allocation, avoiding idle resources and waste; at the same time, through the corresponding server execution of tasks, using advanced algorithms and models (such as time series analysis, machine learning, etc.), it can accurately predict the future power load distribution, which helps to discover and respond to potential load peaks or troughs in advance, and provide a scientific basis for the load management and scheduling of the distribution network. Finally, the predicted future power load distribution is provided to the distribution network, which can assist the distribution network in making more accurate and efficient load management and scheduling decisions, help balance the supply and demand of the power grid, optimize resource allocation, reduce power loss, and ultimately improve the stability and reliability of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0081] Figure 1 The logical block diagram of the distribution network communication method for electric vehicles and 5G base station access.

[0082] Figure 2 This is a network architecture diagram of the terminal device layer, edge computing service layer, and cloud computing layer.

[0083] Figure 3 Flowchart of the improved Q-Learning algorithm.

[0084] Figure 4 This is a communication architecture diagram between the servers in the terminal device layer, edge computing service layer, and cloud computing layer. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0086] It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. In the description of the present invention, it should be noted that the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invention product is usually placed when used, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. In addition, the terms "horizontal", "vertical", etc. do not mean that the components are required to be absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0087] The following is a further detailed description through specific implementation methods:

[0088] Example:

[0089] This embodiment discloses a distribution network communication method for electric vehicles and 5G base station access.

[0090] like Figure 1 As shown, a distribution network communication method for electric vehicles and 5G base station access includes:

[0091] S1: Generate a task set based on the collected data of charging piles and 5G base stations;

[0092] In this embodiment, the relevant data of the charging piles and 5G base stations include the layout data of the charging piles and 5G base stations, as well as the real-time load data, transmission power and / or number of service users collected by sensors.

[0093] S2: Construct a task offloading scheduling model with the goal of minimizing the total cost of executing all tasks in the task set;

[0094] S3: Solve the task offloading scheduling model through the (improved) Q-Learning algorithm to obtain the optimal task offloading scheduling strategy;

[0095] S4: Unload and schedule the tasks in the task set according to the optimal task unloading scheduling strategy, and unload the tasks to the corresponding level of server for execution;

[0096] S5: predict the future power load distribution by executing tasks through servers at the corresponding level;

[0097] S6: Provide the future power load distribution to the distribution network to assist the distribution network in making decisions on load management and scheduling.

[0098] In the present invention, firstly, relevant data of charging piles and 5G base stations (such as charging demand, base station energy consumption, user behavior patterns, etc.) are collected, and a prediction task set is generated through the edge device layer and scheduled through the algorithm, thereby ensuring the effectiveness and pertinence of the task set. Secondly, by constructing a task offloading scheduling model with the goal of minimizing the total cost of executing all tasks in the task set, the costs of computing resources, communication resources, and energy consumption can be considered; at the same time, by solving the task offloading scheduling model with an improved Q-Learning algorithm, it is possible to dynamically adapt to changes in the load of the distribution network, electric vehicles, and 5G base stations and find the optimal task offloading scheduling strategy, so that tasks can be offloaded to the corresponding level of servers for execution while considering the costs of computing resources, communication resources, and energy consumption. This can not only reduce the operating cost of the distribution network, but also improve the overall efficiency and flexibility of the distribution network. Then, according to the optimal task offloading scheduling strategy, the tasks can be automatically and intelligently offloaded to the corresponding level of server execution, ensuring efficient task processing and reasonable resource allocation, avoiding idle resources and waste; at the same time, through the corresponding server execution of tasks, using advanced algorithms and models (such as time series analysis, machine learning, etc.), it can accurately predict the future power load distribution, which helps to discover and respond to potential load peaks or troughs in advance, and provide a scientific basis for the load management and scheduling of the distribution network. Finally, the predicted future power load distribution is provided to the distribution network, which can assist the distribution network in making more accurate and efficient load management and scheduling decisions, help balance the supply and demand of the power grid, optimize resource allocation, reduce power loss, and ultimately improve the stability and reliability of the power grid.

[0099] In order to better introduce the technical solution of the present invention, this embodiment is described through the following parts.

[0100] 1. Network Architecture

[0101] In this embodiment, Figure 2 As shown, the network used to perform tasks includes a three-layer structure: terminal device layer, edge computing (MEC) service layer, and cloud computing layer.

[0102] The servers at the terminal device layer are connected to the sensors of the electric vehicle charging piles and 5G base stations. The servers receive data collected by the sensors to generate computing tasks. The servers scheduled by the edge computing service layer independently run reinforcement learning algorithms, balance task loads through horizontal collaboration, request higher-level task decisions from the cloud computing layer servers when necessary, and optimize their own task allocation using a global perspective. The cloud computing layer uses its grasp of the entire network status to optimize resource scheduling and task allocation through reinforcement learning. In the initial stage, it improves the accuracy of task decisions by providing offloading suggestions to the lower layers. Each connection between the layers has a specific maximum bandwidth.

[0103] In addition, there is vertical collaboration and horizontal collaboration in the network structure: vertical collaboration means that the terminal device layer can upload tasks to the more efficient edge computing service layer for processing, and the edge computing service layer can also hand over tasks to the cloud computing layer for processing when faced with complex computational tasks; horizontal collaboration refers to the collaboration between servers in the edge computing service layer. Servers with larger workloads can hand over part of the tasks to other servers within the range for collaborative execution, thereby achieving a load balancing effect.

[0104] In this embodiment, the optimal task offloading scheduling strategy is used to offload tasks in the task set to servers at the terminal device layer, edge computing service layer or cloud computing layer for execution.

[0105] 2. Load Forecasting Model

[0106] In this embodiment, each task refers to predicting the future power load distribution based on the collected data about charging piles and 5G base stations. Since the coverage range of 5G base stations is small, base stations need to be densely deployed to ensure signal coverage. High-density base station layout areas are usually commercial centers or densely populated areas. The power load in these areas fluctuates greatly, and a refined load forecasting model is required. Therefore, a load forecasting model is constructed and trained based on a long short-term memory network; the trained load forecasting model is deployed on the server; the collected data about charging piles and 5G base stations are input into the trained load forecasting model deployed on the server, and the corresponding future power load distribution is output. When training the load model, the layout and scale data, historical load data, and geographic information data of 5G base stations and charging piles are collected, and duplicate values ​​and outliers are removed to ensure the accuracy and completeness of the data.

[0107] Specifically, the network structure of the long short-term memory network includes:

[0108] 1) Forget Gate

[0109] f t =o(W f ·[h t-1 ,x t ]+b f );

[0110] Where: f t represents the output of the forget gate at the current moment; o is the activation function, which limits the output to the range of [0,1]; W f is the weight matrix of the forget gate, h t-1 is the hidden state of the previous time step, x t is the input vector at the current moment (including base station layout, historical load data, transmission power, number of service users, etc.), b f is the bias of the forget gate;

[0111] 2) Input Gate

[0112] i t =o(W i ·[h t-1 ,x t ]+b i );

[0113] Where: i t represents the output of the input gate, which controls the proportion of new information written; b i represents the corresponding bias term; W i represents the weight matrix of the input gate;

[0114] 3) New candidate memory:

[0115]

[0116] Where: is a candidate new cell state, representing the potential memory content at the current moment; tanh represents the hyperbolic tangent function, which is used for nonlinear transformation; W C represents the weight matrix;

[0117] 4) Update cell status:

[0118]

[0119] Where: C t is the updated cell state;

[0120] 5) Output gate:

[0121] o t =o(W o ·[h t-1 ,x t ]+h o );

[0122] h t =o t *tanh(C i );

[0123] In the formula: o t is the output of the output gate, which controls the output ratio of the cell state; b o is the bias term of the output gate; W o is the weight matrix of the output gate; h t is the hidden state at the current moment.

[0124] 3. Task Offloading Scheduling Model

[0125] In this embodiment, it is assumed that n servers can be represented as a set S = {s1, s2, ...s n}, server. Serversi The tasks are represented as a set T i ={T i1 ,T i1 ,...T ik}, each task contains multiple features, which can be represented as a vector T ik ={a,e,da,dl,m}, which consists of five features: start time a, end time e, input and output data volume da, limit time dl and number of instructions m.

[0126] The indicators of different servers are represented as vector E d ={c,ts,fl,ep,tp,b}, where c is the CPU usage, ts is the current number of tasks, fl is the maximum computing power, ep and tp represent the energy consumption of processing and transmitting each instruction respectively, and b is the bandwidth. d It represents any of the terminal device layer server, edge computing layer server, and cloud computing layer server, which can be written as E l 、E e and E c .

[0127] In this embodiment, the objective function of the task offloading scheduling model is expressed as:

[0128]

[0129]

[0130] Where: G i represents the objective function; Z ik (Δ) represents task T ik The total cost; Δ = 0 means that the task is offloaded to the server of the terminal device layer, Δ = 1 means that the task is offloaded to the server of the edge computing layer, Δ = 2 means that the task is offloaded to the server of the cloud computing layer, and Δ = -1 means that the task transmission fails; Represents task T ik The total cost of server execution at the end device level; Represents task T ik The total cost of executing on edge computing layer servers; Represents task T ik The total cost of server execution at the cloud computing layer.

[0131] The constraints of the task offloading scheduling model include:

[0132] 1) The tasks assigned to the server cannot exceed its maximum computing capacity:

[0133]

[0134] Where: Respectively represent the execution server s i The kth task T generated ik The number of instructions required, the limit time, and the start time; represents the maximum computing capacity of the server; K represents the number of tasks in the task set;

[0135] 2) The total delay of executing the task cannot exceed the time limit:

[0136]

[0137] Where: Indicates the execution of task T ik The total delay.

[0138] 1. Perform tasks at the terminal device layer

[0139] In this embodiment, the task T is calculated by the following steps: ik Total cost of server execution at the end device level

[0140] 1) Computation task T ik Delay caused by server execution at the terminal device layer

[0141]

[0142] Where: Indicates the execution of task T ik The number of instructions required; Indicates the maximum computing capacity of the terminal device layer server;

[0143] 2) Computation task T ik Energy consumption generated by server execution at the terminal device layer

[0144]

[0145] Where: Indicates the energy consumption of each instruction processed by the terminal device layer server;

[0146] 3) Computation task T ik Total cost of server execution at the end device level

[0147]

[0148] Where: α represents the weight of energy consumption.

[0149] 2. Execute tasks at the edge computing layer

[0150] In this embodiment, the task T is calculated by the following steps: ok Total cost of server execution at the edge computing layer

[0151] 1) Computation task T ik The latency caused by server execution at the edge computing layer

[0152]

[0153] Where: Indicates the execution of task T ik The number of instructions required; Indicates the maximum computing capacity of the edge computing layer server; Indicates the execution of task T ik Required amount of input and output data; L ie Indicates server i Maximum transmission rate to edge computing layer servers; represents the bandwidth of the edge computing layer server; S represents the useful signal power; σ 2 is the noise power;

[0154] 2) Computation task T ik Energy consumption generated by server execution at the edge computing layer

[0155]

[0156] Where: Indicates the energy consumption of each instruction transmitted by the edge computing layer server; Indicates the energy consumption of each instruction processed by the edge computing layer server;

[0157] 3) Computation task T ik Total cost of server execution at the edge computing layer

[0158]

[0159] Where: α represents the weight of energy consumption.

[0160] 3. Perform tasks in the cloud computing layer

[0161] In this embodiment, the task T is calculated by the following steps: ok Total cost of server execution at the cloud computing layer

[0162] 1) Computation task T ik The latency caused by server execution in the cloud computing layer

[0163]

[0164] Where: Indicates the execution of task T ik The number of instructions required; Indicates the maximum computing capacity of the cloud computing layer server; Represents task T ik Required amount of input and output data; L ic For servers i Maximum transfer rate to cloud computing layer servers; represents the bandwidth of the cloud computing layer server; S represents the useful signal power; σ 2 is the noise power;

[0165] 2) Computation task T ik Energy consumption generated by server execution at the cloud computing layer

[0166]

[0167] Where: It represents the energy consumption of each instruction transmitted by the cloud computing layer server; It represents the energy consumption of each instruction processed by the cloud computing layer server;

[0168] 3) Computation task T ik Total cost of server execution at the cloud computing layer

[0169]

[0170] Where: α represents the weight of energy consumption.

[0171] 4. Improved Q-Learning Algorithm

[0172] The Q-Learning algorithm is an offline reinforcement learning algorithm that learns how to perform best in a controllable Markov domain by maintaining a value table and selecting the action with the highest value through the value table.

[0173] In this embodiment, the Q-Learning algorithm is improved. Figure 3 As shown, the processing steps of the improved Q-Learning algorithm are as follows:

[0174] S401: construct a Q table to store the Q value of each action taken in each state;

[0175] S402: Setting the learning rate β, discount factor γ, exploration rate ε, penalty factor ζ, decay coefficient ω, value coefficient η of the algorithm; initial environment state;

[0176] In this embodiment, the environmental status includes the allowed latency, CPU usage, and processing speed of the server. The allowed latency, CPU usage, and processing speed are discretized: when the allowed latency is less than 6s, it is recorded as "low", when the allowed latency is between 6s-15s, it is recorded as "medium", and when the allowed latency is greater than 15s, it is recorded as "high". When the CPU usage is between 0-25%, it is recorded as "low"; when it is between 25%-50%, it is recorded as "medium"; 50%-75% is recorded as "high"; 75%-100% is recorded as "very high". The processing speed is less than 3×10 per second. 10 instructions per second, recorded as "low", the processing speed is less than 10310× instructions per second, recorded as "low", 3×10 10 -1.2×10 11 , greater than 1×10 11 Recorded as "high".

[0177] S403: For each task t i ∈Task Set T i , perform the following steps:

[0178] S404: Obtain the current environment state CS; determine the current optional action set A according to the environment state CS; select action a from the action set A based on the ε-greedy strategy; combine Figure 4 As shown, action a is executed at the terminal device layer, offloaded to the edge computing layer for execution, offloaded to the cloud computing layer for execution, or querying the cloud computing layer server (hereinafter also referred to as the cloud server); Δ=0 means offloading to the terminal device layer server, Δ=1 means offloading to the edge computing layer server, Δ=2 means assigned to the cloud computing layer, and Δ=3 means querying the cloud server upward.

[0179] In this embodiment, the cloud server can obtain real-time working status data more quickly and have a more accurate grasp of global knowledge. Therefore, the servers in the cloud computing layer can also run their own reinforcement learning algorithms. Although they are the same as the algorithms executed by the edge computing layer servers, their different understandings of the environment will lead to different final decisions. In the early stage of the algorithm when knowledge is insufficient, the uninstallation decision is delegated to the cloud server. In this way, while obtaining more accurate global information, knowledge can be used to assist the current device in making uninstallation choices. In order to increase the frequency of upward queries in the early stage of the algorithm, reducing the frequency of upward queries in the middle and late stages can effectively solve this problem. Therefore, an attenuation coefficient is introduced. After each upward query, this coefficient is used to increase the cost of the behavior.

[0180] S405: Execute action a and obtain a new environment state CS′ after the execution of action a;

[0181] S406: Calculate the reward according to the environment state CS and the environment state CS′ in combination with the objective function of the task offloading scheduling model;

[0182] 1) When the action is executed at the terminal device layer, offloaded to the edge computing layer, or offloaded to the cloud computing layer, the reward is calculated using the following formula:

[0183]

[0184] Where: Z i,k Indicates the execution of task T ik Rewards; Indicates the execution of task T ik The delay generated Indicates the execution of task T ik The energy consumption generated, where the superscript d = l, e, c, respectively, represents the server of the terminal device layer, edge computing service layer or cloud computing layer; Represents task T ik time limit;

[0185] 2) When the action is to query the cloud computing layer server upward, the reward is calculated by the following formula:

[0186] Z i,k ″=Z i,k ′·ω;

[0187] Z i,k ′=Z i,k ·η; The purpose of this formula is to reduce the η cost function by the value coefficient, making the decisions made by the cloud computing layer server more valuable.

[0188] Where: Z i,k ″ indicates the reward for upward query; Z i,k Indicates the execution of task T ik The reward is ; η is the value coefficient; ω is the decay coefficient. After each upward query, the decay coefficient ω needs to be increased. The purpose is to control the cost of upward query through the decay coefficient ω. In the early stage, the cost of upward query is low, which encourages devices to rely on global information to quickly optimize strategies. In the later stage, by increasing the query cost (controlled by ω\omegaω), devices are encouraged to make independent decisions and reduce unnecessary queries.

[0189] S407: Calculate the current Q value according to action a, environment state CS′ and reward Z, and update the Q table;

[0190] The calculation formula of Q value is:

[0191]

[0192] Where: a t、cs t , Z t Represent the action, environment state and reward at time t respectively;

[0193] S408: Repeat steps S403 to S407 until all tasks are completed and the final Q table is output;

[0194] S409: Selecting the most valuable offloading action for each task based on the Q table to form an optimal task offloading scheduling strategy.

[0195] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. A distribution network communication method for electric vehicles and 5G base station access, characterized in that: include: S1: Generate a task set based on the collected data of charging piles and 5G base stations; S2: Construct a task offloading scheduling model with the goal of minimizing the total cost of executing all tasks in the task set; S3: Solve the task offloading scheduling model through the Q-Learning algorithm to obtain the optimal task offloading scheduling strategy; S4: Unload and schedule the tasks in the task set according to the optimal task unloading scheduling strategy, and unload the tasks to the corresponding level of server for execution; S5: predict the future power load distribution by executing tasks through servers at the corresponding level; S6: Provide the future power load distribution to the distribution network to assist the distribution network in making decisions on load management and scheduling.

2. The distribution network communication method for electric vehicles and 5G base station access as claimed in claim 1, characterized in that: In step S1, the task is to predict the future power load distribution based on the collected data on charging piles and 5G base stations.

3. The distribution network communication method for electric vehicles and 5G base station access as claimed in claim 2, characterized in that: In step S5, a load forecasting model is constructed and trained based on the long short-term memory network; the trained load forecasting model is deployed on the server; the collected relevant data of charging piles and 5G base stations are input into the trained load forecasting model deployed on the server, and the corresponding future power load distribution is output.

4. The distribution network communication method for electric vehicles and 5G base station access as claimed in claim 1, characterized in that: In step S3, the optimal task offloading scheduling strategy is used to offload the tasks in the task set to the server at the terminal device layer, edge computing service layer or cloud computing layer for execution.

5. The distribution network communication method for electric vehicles and 5G base station access as claimed in claim 4, characterized in that: In step S3, the objective function of the task offloading scheduling model is expressed as: Where: G i represents the objective function; Z ik (Δ) represents task T ik The total cost; Δ = 0 means that the task is offloaded to the server of the terminal device layer, Δ = 1 means that the task is offloaded to the server of the edge computing layer, Δ = 2 means that the task is offloaded to the server of the cloud computing layer, and Δ = -1 means that the task transmission fails; Represents task T ik The total cost of server execution at the end device level; Represents task T ik The total cost of executing on edge computing layer servers; Represents task T ik The total cost of server execution at the cloud computing layer.

6. The distribution network communication method for electric vehicles and 5G base station access as claimed in claim 5, characterized in that: In step S3, the constraints of the task offloading scheduling model include: 1) The tasks assigned to the server cannot exceed its maximum computing capacity: Where: Respectively represent the server s i The kth task T generated ik The number of instructions required to complete, the limit time, and the start time; represents the maximum computing capacity of the server; K represents the number of tasks in the task set; 2) The total delay of executing the task cannot exceed the time limit: Where: Indicates the execution of task T ik The total delay of 7. The distribution network communication method for electric vehicles and 5G base station access as claimed in claim 5, characterized in that: In step S3, the task T is calculated by the following steps: ik Total cost of server execution at the end device level 1) Computation task T ik Delay caused by server execution at the terminal device layer Where: Indicates the execution of task T ik The number of instructions required; Indicates the maximum computing capacity of the terminal device layer server; 2) Computation task T ik Energy consumption generated by server execution at the terminal device level Where: Indicates the energy consumption of each instruction processed by the terminal device layer server; 3) Computation task T ik Total cost of server execution at the end device level Where: α represents the weight of energy consumption.

8. The distribution network communication method for electric vehicles and 5G base station access as claimed in claim 5, characterized in that: In step S3, the task T is calculated by the following steps: ik Total cost of server execution at the edge computing layer 1) Computation task T ik The latency caused by server execution at the edge computing layer Where: Indicates the execution of task T ik The number of instructions required; Indicates the maximum computing capacity of the edge computing layer server; Indicates the execution of task T ik Required amount of input and output data; L ie Indicates the generation of task T ik Servers i Maximum transmission rate to edge computing layer servers; represents the bandwidth of the edge computing layer server; S represents the useful signal power; σ 2 is the noise power; 2) Computation task T ik Energy consumption generated by server execution at the edge computing layer Where: Represents the energy consumption of each instruction transmitted by the edge computing layer server; Indicates the energy consumption of each instruction processed by the edge computing layer server; 3) Computation task T ik Total cost of server execution at the edge computing layer Where: α represents the weight of energy consumption.

9. The distribution network communication method for electric vehicles and 5G base station access as claimed in claim 5, characterized in that: In step S3, the task T is calculated by the following steps: ik Total cost of server execution at the cloud computing layer 1) Computation task T ik The latency caused by server execution in the cloud computing layer Where: Indicates the execution of task T ik The number of instructions required; Indicates the maximum computing capacity of the cloud computing layer server; Represents task T ik Required amount of input and output data; L ic For servers i Maximum transfer rate to cloud computing layer servers; represents the bandwidth of the cloud computing layer server; S represents the useful signal power; σ 2 is the noise power; 2) Computation task T ik Energy consumption generated by server execution at the cloud computing layer Where: It represents the energy consumption of each instruction transmitted by the cloud computing layer server; It represents the energy consumption of each instruction processed by the cloud computing layer server; 3) Computation task T ik Total cost of server execution at the cloud computing layer Where: α represents the weight of energy consumption.

10. The distribution network communication method for electric vehicles and 5G base station access as claimed in claim 6, characterized in that: In step S4, the Q-Learning algorithm is improved; the processing steps of the improved Q-Learning algorithm are as follows: S401: construct a Q table to store the Q value of each action taken in each state; S402: Setting the learning rate β, discount factor γ, exploration rate ε, penalty factor ζ, decay coefficient ω, value coefficient η of the algorithm; initial environment state; S403: For each task t i ∈Task Set T i , perform the following steps: S404: Obtain the current environment state CS; determine the currently available action set A according to the environment state CS; select action a from the action set A based on the ε-greedy strategy; wherein action a is executed at the terminal device layer, offloaded to the edge computing layer, offloaded to the cloud computing layer, or querying the cloud computing layer server upward; S405: Execute action a and obtain a new environment state CS′ after the execution of action a; S406: Calculate the reward according to the environment state CS and the environment state CS′ in combination with the objective function of the task offloading scheduling model; 1) When the action is executed at the terminal device layer, offloaded to the edge computing layer, or offloaded to the cloud computing layer, the reward is calculated using the following formula: Where: Z i,k Indicates the execution of task T ik Rewards; Indicates the execution of task T ik The delay generated, Indicates the execution of task T ik The energy consumption generated, where the superscript d = l, e, c, respectively, represents the server of the terminal device layer, edge computing service layer or cloud computing layer; Represents task T ik time limit; 2) When the action is to query the cloud computing layer server upward, the reward is calculated by the following formula: WITH i,k ″=Z i,k ′ ω; WITH i,k ′=Z i,k ·η; Where: Z i,k ″ indicates the reward for upward query; Z i,k Indicates the execution of task T ik Rewards; η represents the value coefficient; ω represents the attenuation coefficient, where the attenuation coefficient needs to be increased after each upward query; S407: Calculate the current Q value according to action a, environment state CS′ and reward Z, and update the Q table; The calculation formula of Q value is: Where: a t 、cs t , Z t Represent the action, environment state and reward at time t respectively; S408: Repeat steps S403 to S407 until all tasks are completed and the final Q table is output; S409: Selecting the most valuable offloading action for each task based on the Q table to form an optimal task offloading scheduling strategy.

Images