Vehicle-mounted edge computing network scheduling method based on information age

Abstract

The invention relates to a vehicle-mounted edge computing network scheduling method based on information age, and belongs to the field of mobile edge computing. The method is oriented to a vehicle-mounted edge computing network in which a fixed server and a vehicle-mounted mobile server coexist, and aims to minimize the long-term average information age of the system by selecting a device node combination scheduled by the server in each time slot. The position constraint, the link conflict constraint and the server computing resource constraint of a vehicle-mounted mobile server are considered, an optimization problem is modeled as a Markov decision process, and a deep breach Q network is improved by using a bifurcation structure and an action mask to obtain a scheduling strategy of an equipment node. According to the method, the bifurcation structure and the action mask are introduced into the deep breach Q network, so that the problem solving complexity is reduced, and the data delivery freshness of the vehicle-mounted edge computing network system is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of mobile edge computing and relates to a vehicle-mounted edge computing network scheduling method based on information age. Background Art

[0002] With the continuous development of edge computing technology, it has achieved significant improvements in data processing and transmission speed, which not only enhances the ability to process data in real time, but also greatly promotes the popularization and development of real-time application technologies such as smart factories and Internet of Vehicles. In such computing-intensive and delay-sensitive application scenarios, the freshness of data is particularly important. Therefore, the age of information (AoI) has been proposed as a new data freshness metric. Unlike traditional indicators such as latency, the age of information comprehensively considers factors such as transmission time and sending interval, providing a comprehensive measure of the freshness of data received at the destination.

[0003] In the vehicle-mounted edge computing scenario, vehicles such as smart mining vehicles and smart logistics vehicles are usually equipped with abundant computing resources. These devices can not only efficiently process their own data and tasks, but also provide computing services for surrounding device nodes with limited computing power when idle. In the IoT system assisted by vehicle-mounted edge computing, the fixed servers deployed at the base station and the vehicle-mounted mobile servers jointly provide computing services for the devices in the network, and minimize the information age of the system through reasonable task scheduling and resource allocation to ensure the freshness of the data. However, due to the different computing power of the fixed server and the vehicle-mounted mobile server, and the large difference in transmission distance, how to dynamically select the server to process the computing task is a difficult problem. In addition, the dynamic mobility characteristics of the vehicle-mounted mobile server further aggravate the complexity of the scheduling strategy design. Therefore, how to invent a scheduling method for device nodes to minimize the long-term average information age of the system and improve the information freshness of the system under the constraints of channels, computing resources and the location of the vehicle-mounted mobile server has become an important challenge. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a vehicle-mounted edge computing network scheduling method based on information age. For the scenario where fixed servers and vehicle-mounted mobile servers exist in the vehicle-mounted edge computing network system, the computing resources of the edge computing server are comprehensively considered, and the computing tasks of the device nodes are scheduled to the edge computing server for processing. The present invention constructs a vehicle-mounted edge computing-assisted computing network information age update model, and expresses the optimization problem of minimizing the long-term average information age of the system as a Markov decision process. In order to solve this optimization problem, the present invention further introduces a bifurcation structure and an action mask, thereby improving the traditional deep reinforcement learning algorithm. This improvement not only reduces the state space and action space in the deep reinforcement learning method, but also speeds up the learning speed of the scheduling method, thereby improving the data freshness of the system.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A scheduling method for vehicle-mounted edge computing networks based on information age. In a computing network system model where fixed servers and vehicle-mounted mobile servers coexist, a deep reinforcement learning scheduling algorithm improved by bifurcation structure and action mask is used to optimize its scheduling strategy in a system model where two servers provide computing services. That is, the computing tasks of device nodes are scheduled to idle server nodes for computing processing in each time slot. The long-term average information age of the system is minimized while satisfying the resource constraints of edge computing servers, link conflict constraints, and the location constraints of vehicle-mounted mobile servers.

[0007] The method specifically comprises the following steps:

[0008] S1: Obtain relevant parameter information of the vehicle-mounted edge computing network system, build the system's communication model, task scheduling model, and information age update model, and build a scheduling network for learning the optimal scheduling strategy;

[0009] S2: Construct an optimization problem to minimize the long-term average information age of the system, and establish the state space, action space and reward function of the system based on the communication model, task scheduling model and information age update model, so as to transform the strongly coupled optimization problem of server computing resources, link conflicts and vehicle-mounted mobile server location into a Markov decision process;

[0010] S3: Update the system state in each time slot, optimize the action space of the neural network through shared decision modules and bifurcation structures, and introduce an action mask mechanism to assign a mask to the action Q value in a specific state, thereby further reducing the number of optional action spaces and improving the convergence speed and robustness of the algorithm;

[0011] S4: Train the scheduling network, calculate the loss function, and then update the network parameters through the gradient descent algorithm until the network converges, and finally obtain a link scheduling strategy suitable for the vehicle-mounted mobile server-assisted IoT device system.

[0012] Further, in step S1, a communication model of the system is constructed, which specifically includes the following steps:

[0013] S101: Consider an IoT system consisting of M server nodes, including M-1 vehicle-mounted mobile servers and one fixed server, which are used to provide computing services for N device nodes. The computing tasks generated by the device nodes are transmitted to the server nodes through wireless channels for processing, and the computing results are returned to the corresponding device nodes. Compared with fixed servers, vehicle-mounted mobile servers are mobile and can be closer to device nodes, thereby improving the transmission quality of wireless channels. The server index is recorded as Where m = 0 represents a fixed server; in addition, a separate vehicle-mounted mobile server set is defined as Likewise, the index of a device node is defined as The location of the device node is fixed; the system processing task is based on the time slot. Represents the index of the time slot; at the beginning of each time slot, the system can schedule at most min{M,N} device nodes to transmit data packets through the orthogonal channel. Considering the distortion and thermal noise in the wireless channel, we assume that the data transmission is not always successful. After the data packet is successfully transmitted, it will be calculated and processed on the corresponding server;

[0014] S102: Set the fixed server as the coordinate origin to establish a coordinate system, whose coordinates are G m (t) = (0,0), the device node coordinates can be established as The coordinates of the vehicle-mounted mobile server can be expressed as At time slot t, the Euclidean distance d between device node n and server node m n,m (t) is:

[0015]

[0016] in, and (x m (t),y m (t)) represent the location coordinates of device node n and server node m respectively;

[0017] S103: Wireless channel gain h between device node n and server node m n,m (t) is:

[0018] h n,m (t) = h0(d n,m (t))-2

[0019] Where, h0 is the unit power gain of the channel;

[0020] S104: According to Shannon's formula, the uplink transmission rate R from device node n to server node m n,m (t) is:

[0021]

[0022] in, is the Gaussian noise power, B is the bandwidth, is the transmission power of device node n, h n,m (t) is the wireless channel gain from device node n to server node m.

[0023] Further, in step S1, a task scheduling model of the system is constructed, which specifically includes the following steps:

[0024] S111: Assume that each computing task can only be scheduled by one server node, and that a single server node cannot schedule computing tasks for multiple device nodes in the same time slot; since the amount of data of the computing result is generally small, the transmission time from the server node to the device node can be ignored; the device node generates computing tasks (composed of data packets) according to the parameter λ n ∈(0,1] Bernoulli distribution; Each device node uses a buffer to store data packets. When a new data packet is generated, the old data packet in the buffer will be replaced; The server node must complete the calculation task of the current device node before scheduling other device nodes; The finite positive integer c n,m (t) represents the total number of time slots required for server node m to complete the computing task of device node n in the current time slot t;

[0025] S112: The present invention considers two types of task offloadable objects, namely, offloading data packets to a fixed server or a vehicle-mounted mobile server for calculation; in order to better describe the scheduling strategy, O n (t) = {0, 1, 2} to represent the task offloading decision of device node n in time slot t, specifically: n (t) = 0 means that the server node does not schedule the device node n in time slot t; n (t) = 1 means that the device node n is scheduled to the vehicle-mounted mobile server for computing and processing in time slot t; n (t) = 2 means that the device node n is scheduled to the fixed server for computing processing in time slot t;

[0026] When uninstall decision O nWhen (t) = 1, the device node n chooses to offload the computing task to the nearby vehicle-mounted mobile server. Since the computing task offloading in the actual system usually takes a short time, the moving distance of the vehicle can be ignored; that is, the uplink transmission rate remains constant during the offloading process, and the total time required to complete the computing task is calculated as:

[0027]

[0028] in, represents the total time required for device node n to offload the computing task to the vehicle-mounted mobile server m in time slot t, D n (t) represents the computing task of device node n in time slot t, R n,m (t) represents the uplink transmission rate from device node n to server node m at time slot t, C represents the number of CPU cycles required to process 1 bit of data, represents the calculation frequency of the vehicle-mounted mobile server m;

[0029] When uninstall decision O n (t) = 2, the device node n offloads the computing task to the fixed server for computing. Since the device node and the fixed server are fixed in position, the uplink transmission rate will not change, so the total time required to complete the computing task is calculated as:

[0030]

[0031] in, represents the total time required for device node n to offload computing tasks to a fixed server in time slot t, Indicates the calculation frequency of a fixed server.

[0032] Further, in step S1, a system information age update model is constructed, which specifically includes the following steps:

[0033] S121: Information age is the time that has passed since the destination received the latest data packet. Considering that the information age of the buffer will affect the information age of the destination, let g n (t)∈{0,1} indicates whether device node n generates a new task in time slot t. If so, g n (t)=1, otherwise g n (t) = 0; the information age of the data packet in the buffer The expression is as follows:

[0034]

[0035] S122: The information age of the data at the target node is affected by the transmission time and the server calculation time. According to the total time required for the device node n to offload the task to the server node m under different scheduling methods, the rounding up function is used. Discretize the total time into the total number of time slots c n,m (t) is:

[0036]

[0037] In addition, define l n (t)∈{0,1,…,c n,m (t)} represents the number of data packets remaining from task n of device node calculated by the server node in time slot t; let i m (t)∈{0,1} represents the state of the server in time slot t, where i m (t) = 1 means server m is in computing state, otherwise i m (t) = 0; l n The updating process of (t) is:

[0038]

[0039] Among them, y n,m (t)∈{0,1} represents the state of device node n being scheduled by server m at the beginning of time slot t, y n,m (t) = 1 means that the computing task of device node n is scheduled to server node m, otherwise y n,m (t) = 0; k n,m (t)∈{0,1} indicates whether the device node n successfully sends the data packet to the server node m;

[0040] S123: When the server node completes the computing task, it transmits the result back to the corresponding device node; represents the information age of the destination device node n, and The update process is expressed as:

[0041]

[0042] Among them l n (t) represents the number of data packets remaining from task n of device node calculated by the server node in time slot t, i m (t) represents the state of server m at time slot t.

[0043] Further, step S2 specifically includes the following steps:

[0044] S21: Under the condition of long-term time T→∞, the data freshness of the network is measured by the average information age of the device node at the destination. Under any feasible strategy π, the expected average information age of the device node at the destination is It is expressed as:

[0045]

[0046] in, Indicates the initialization state information age of the system;

[0047] S22: Define Π to represent the set of all feasible scheduling strategies, where π∈Π, π * is the optimal scheduling strategy; the optimization problem of minimizing the long-term average information age of the system is expressed as:

[0048]

[0049] in, represents the location set of the vehicle-mounted mobile server at time slot t, represents the location set of the device node in time slot t, G m＝0 (t) represents the position of the fixed server at time slot t, R represents the maximum radius at which the fixed server can establish a connection, r represents the maximum radius at which the vehicle-mounted mobile server can establish a connection, ||·|| 2 represents the Euclidean distance function;

[0050] S23: Establish the optimization problem in step S22 as a Markov decision process; let v n (t)∈{0,1,…,M-1} represents the index of the optimal offloading server node of device node n at time slot t, v n (t) = 0 means that there is no vehicle-mounted mobile server that can be unloaded around device node n in time slot t, and it can only be unloaded to the fixed server; the state space of the system includes the information age of the data packets of all device nodes, the operating status of the server, and the location information of the vehicle-mounted mobile server; therefore, the state space of the system is established as:

[0051]

[0052] in, v(t)=(v0(t),…,v N-1 (t)), i(t)=(i0(t),…,i M-1 (t)), l(t)=(l0(t),…,l N-1 (t));

[0053] S24: The server node in the system can select a device node for scheduling. Let a m(t)∈{-1,0,1,…,N-1} represents the device node index scheduled by server node m at time slot t; where a m (t) = -1 means that the server node m is not currently scheduling any device node, a m (t)≥0 indicates that server node m schedules the index of device node n in time slot t. The action combination of all server nodes constitutes the action of the system, which can be specifically expressed as:

[0054] a(t)=(a0(t),…,a M-1 (t))

[0055] For simplicity, the matrix ω(t) with dimension N×M is set to represent the link pairs of the server node scheduling device nodes to indicate the scheduling status of the current time slot t, thus obtaining:

[0056]

[0057] Among them, y n,m (t)∈{0,1} represents the indicator function of whether the computing task of device node n is scheduled to server node m or not. If y n,m (t) = 1, which means that the computing task of device node n is scheduled to server node m, and we can get a m (t) = n;

[0058] S25: Establish the reward function r(t) of the system as:

[0059]

[0060] Among them, r b The benchmark value of the long-term average information age can be expressed by Conclude.

[0061] Further, step S3 specifically includes the following steps:

[0062] S31: The neural network structure of Dueling DQN with improved bifurcation structure optimizes the action space from exponential growth to linear growth, including: dividing the network into M bifurcation structures corresponding to the number of server nodes, each bifurcation structure is assigned to a specific server node; the output dimension of each bifurcation structure corresponds to the number of executable sub-actions N+1 on its server; in addition, the value function network and the advantage function network are set in the deep network Q network, and the advantage function of the bifurcation sub-action is expressed by H(s,a m |θ) represents, V(s|θ) represents the state estimate of the online network; therefore, the Q value Q of the sub-action in the online network m (s,a|θ) can be expressed as:

[0063]

[0064] in, represents any executable sub-action of the fork structure, θ t represents the parameters of the online network at time slot t;

[0065] S32: further introduce action mask to prevent invalid action selection, specifically including: after calculating the Q value of each sub-action, the algorithm verifies whether the device node is within the signal coverage of the server according to the location information of the vehicle-mounted mobile server, and checks whether the current channel state is suitable for unloading. If the device node n does not meet the unloading conditions, a smaller value is assigned to the Q value of n in the fork m. Therefore, the Q value of fork m can be updated as:

[0066]

[0067] Among them, v n (t) represents the optimal unloading object of device node n in time slot t, Q m (s(t),a m (t)|θ t ) represents the Q value of the forked sub-action obtained through the online network.

[0068] Further, step S4 specifically includes the following steps:

[0069] S41: After step S3, an improved Dueling DQN network is obtained, which includes an online network and a target network. Both networks include a shared decision module and a deep Q network with a multi-fork structure improvement. The algorithm inputs the system state space. According to the ε-greedy strategy, the agent selects the action combination with the largest Q value with a probability of 1-ε, and randomly selects an action combination from the action space with a probability of ε. After executing the action, (s(t), a(t), r(t), s(t+1)) is stored in the experience replay pool to form an experience.

[0070] S42: Randomly sample batch size experience in the experience replay pool for training. Then, according to step S31, the online network subtracts the average advantage of all sub-actions in the fork from the advantage of the sub-action and combines it with the state value to obtain the Q value of the sub-action. Since the network structure of the target network Q′ and the online network Q is the same, the forked target network and its parameters are defined as Q' respectively. m (s,a' m |θ') and θ', for each bifurcation, the target value y m (t) can be calculated as:

[0071]

[0072] Among them, r(t) represents the timely reward, γ represents the discount factor, and a' m is any optional sub-action of forking m in the online network;

[0073] S43: According to the estimated value and the target value returned by the target value network, the loss function Loss is obtained as:

[0074]

[0075] Where M represents the number of forks, y m (t) represents the target value, Q m (s(t),a m (t)|θ t ) represents the Q value of the online network;

[0076] S44: Based on the loss function of step S43, the gradient loss function is calculated for:

[0077]

[0078] in, Represents the estimated network parameter θ t The gradient vector, Q m (s(t),a m (t)|θ t ) represents the Q value of the fork;

[0079] S45: Every several steps, the network parameters of the estimated network are updated to the target network until the network converges.

[0080] The beneficial effects of the present invention are:

[0081] (1) Based on comprehensive consideration of constraints such as edge server computing resources, link conflicts, and the location of the vehicle-mounted mobile server, the present invention optimizes the long-term average information age of the system by training an autonomous learning scheduling method, thereby improving the freshness of the system data delivery.

[0082] (2) The present invention improves the deep dueling Q network by introducing a multi-fork structure, reducing the state space and action space in the network representation. In addition, the adoption of the action mask mechanism enhances the stability of the training process, thereby significantly improving the learning speed of the scheduling method. This enables the scheduling method to be efficiently deployed in the vehicle-mounted edge computing network.

[0083] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0085] Figure 1 It is a schematic diagram of the structure of the computing network scheduling system assisted by vehicle-mounted edge computing based on information age of the present invention;

[0086] Figure 2 A schematic diagram of the deep duel Q network and action mask mechanism with improved bifurcated structure adopted by the present invention;

[0087] Figure 3 This is a flow chart of the vehicle-mounted edge computing network scheduling method based on information age of the present invention. DETAILED DESCRIPTION

[0088] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0089] See also Figure 1 to Figure 3 The present invention aims at a computing network system assisted by an on-board edge computing server, comprehensively considers constraints such as server computing resources, link conflicts, and the location of on-board mobile servers, and minimizes the long-term average information age of the system. The deep reinforcement learning method with improved bifurcation structure is adopted to divide the large action space originally caused by multiple edge server nodes into bifurcated small action spaces divided by the number of server nodes, thereby reducing the dimension of the action space, accelerating the algorithm training speed, and obtaining a scheduling method for a computing network system assisted by an on-board edge computing server. The present invention further introduces the mechanism of action masking, enhances the stability of the training process, and accelerates the learning speed of the scheduling method.

[0090] Figure 1Schematic diagram of the computing network system structure assisted by the vehicle-mounted edge computing server. The system is an IoT system where M server nodes (including M-1 vehicle-mounted mobile servers and one fixed server) provide computing services for N device nodes. The computing tasks generated by the device nodes are transmitted to the server nodes through wireless channels for processing, and the computing results are returned to the corresponding device nodes. Compared with fixed servers, vehicle-mounted mobile servers can be closer to device nodes, thereby improving the quality of wireless channels. The server index is denoted as Where m = 0 represents a fixed server. In addition, a separate vehicle-mounted mobile server set is defined as Likewise, the index of a device node is defined as And the location of the device node is fixed.

[0091] The system processes computing tasks based on time slots, and Represents the index of the time slot. At the beginning of each time slot, the system can schedule at most min{M,N} device nodes to transmit data packets through orthogonal channels. Considering the distortion and thermal noise in the wireless channel, we assume that data transmission is not always successful. After the data packet is successfully transmitted, it will be calculated and processed on the corresponding server. Since the amount of calculation result data is generally small, the transmission time from the server node to the device node can be ignored.

[0092] Figure 2 The schematic diagram of the improved Q network and action mask mechanism of the bifurcation structure used in the present invention is mainly composed of a shared decision module, a multi-bifurcation structure and an action mask. The network is divided into M bifurcation structures, each of which is assigned to a specific server node. The output dimension of each bifurcation structure corresponds to the number of executable sub-actions of the server node N+1. Let represents any executable sub-action of the fork structure, a m =-1 means the server node does not perform scheduling, a m =n represents the device node n scheduled by the server node m. The Q value of the sub-action can be obtained by adding the advantage value of the sub-action to the state value and then subtracting the average advantage of all sub-actions in the fork:

[0093]

[0094] Among them, H(s,a m |θ) represents the advantage value of the sub-action, and V(s|θ) represents the state estimation value of the online network.

[0095] After calculating the Q value of each sub-action, verify whether the device node is within the signal coverage of the server according to the location information of the vehicle mobile server, and check whether the current channel state is suitable for unloading. If the device node n does not meet the unloading conditions, assign a smaller value to the Q value of n in the fork m. Therefore, the Q value of fork m can be updated as:

[0096]

[0097] Figure 3 Flow chart of the computing network scheduling method assisted by vehicle-mounted edge computing based on information age of the present invention, such as Figure 3 As shown, the specific steps include:

[0098] V1~V4: Obtain the computing network system model parameters assisted by the on-board edge computing server, initialize the parameters in the network, and determine the network's state space, action space, and reward function.

[0099] V5~V6: Get the location of the vehicle-mounted mobile server, calculate the theoretical time slots for each device node to complete the calculation under different offloading objects, and update the state space of the system.

[0100] V7~V11: Use the ε-greedy action exploration strategy. If the generated ε is greater than the set threshold ε greedy , then randomly select an action output in the action space, otherwise select the action with the largest Q value as the output, and execute the scheduling decision in the current state, calculate the reward function, and store the current system state, action, reward and next state as experience in the experience replay pool.

[0101] V12~V16: Randomly select experience sets from the experience replay pool to form training segments, calculate the estimated value and target value of each fork, and assign action masks to Q values ​​through the action mask mechanism. Then calculate the loss function through the estimated value and target value, calculate the gradient and update the network weights, and update the parameters of the estimated network to the target network every several steps.

[0102] V17~V19: After determining whether the loss function is stable and has reached the training termination condition, if the termination condition has not been reached, execute V5. If the termination condition has been reached, link scheduling is unloaded according to the fully trained scheduling network.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A vehicle-mounted edge computing network scheduling method based on information age, characterized in that: In a computing network system model where fixed servers and on-vehicle mobile servers coexist, a deep reinforcement learning scheduling algorithm improved by bifurcation structure and action mask is used to optimize its scheduling strategy in a system model where two servers provide computing services. That is, the computing tasks of device nodes are scheduled to idle server nodes for computing processing in each time slot, and the long-term average information age of the system is minimized while satisfying the resource constraints of edge computing servers, link conflict constraints, and the location constraints of on-vehicle mobile servers. The method specifically comprises the following steps: S1: Obtain relevant parameter information of the vehicle-mounted edge computing network system, build the system's communication model, task scheduling model, and information age update model, and build a scheduling network for learning the optimal scheduling strategy; S2: Construct an optimization problem to minimize the long-term average information age of the system, and establish the state space, action space and reward function of the system based on the communication model, task scheduling model and information age update model, so as to transform the strongly coupled optimization problem of server computing resources, link conflicts and vehicle-mounted mobile server location into a Markov decision process; S3: Update the system state in each time slot, optimize the action space of the neural network through the shared decision module and bifurcation structure, and introduce the action mask mechanism to assign a mask to the action Q value in a specific state; S4: Train the scheduling network, calculate the loss function, and then update the network parameters through the gradient descent algorithm until the network converges, and finally obtain a link scheduling strategy suitable for the vehicle-mounted mobile server-assisted IoT device system.

2. The vehicle-mounted edge computing network scheduling method according to claim 1, characterized in that: In step S1, a communication model of the system is constructed, which specifically includes the following steps: S101: Consider an IoT system consisting of M server nodes, including M-1 vehicle-mounted mobile servers and one fixed server, which are used to provide computing services for N device nodes. The computing tasks generated by the device nodes are transmitted to the server nodes through wireless channels for processing, and the computing results are returned to the corresponding device nodes. Compared with fixed servers, vehicle-mounted mobile servers are mobile and can be closer to device nodes, thereby improving the transmission quality of wireless channels. The server index is recorded as Where m = represents a fixed server; in addition, a separate vehicle-mounted mobile server set is defined as Likewise, the index of a device node is defined as The location of the device node is fixed; the system processing task is based on the time slot. Represents the index of the time slot; at the beginning of each time slot, the system can schedule at most min{M,N} device nodes to transmit data packets through the orthogonal channel. Considering the distortion and thermal noise in the wireless channel, it is assumed that the data transmission is not always successful. After the data packet is successfully transmitted, it will be calculated and processed on the corresponding server; S102: Set the fixed server as the coordinate origin to establish a coordinate system, whose coordinates are G m (t)=(0,0), the device node coordinates are established as The coordinates of the vehicle-mounted mobile server are expressed as At time slot t, the Euclidean distance d between device node n and server node m n,m (t) is: in, and (x m (t),y m (t)) represent the location coordinates of device node n and server node m respectively; S103: Wireless channel gain h between device node n and server node m n,m (t) is: h n,m (t)=h0(d n,m (t)) -2 Where, h0 is the unit power gain of the channel; S104: According to Shannon's formula, the uplink transmission rate R from device node n to server node m n,m (t) is: in, is the Gaussian noise power, B is the bandwidth, is the transmission power of device node n, h n,m (t) is the wireless channel gain from device node n to server node m.

3. The vehicle-mounted edge computing network scheduling method according to claim 2, characterized in that: In step S1, a task scheduling model of the system is constructed, which specifically includes the following steps: S111: Assume that each computing task can only be scheduled by one server node, and that a single server node cannot schedule computing tasks for multiple device nodes in the same time slot; the transmission time from the server node to the device node is negligible; the computing task composed of data packets generated by the device node follows the parameter λ n ∈(0,1] Bernoulli distribution; Each device node uses a buffer to store data packets. When a new data packet is generated, the old data packet in the buffer will be replaced; The server node must complete the calculation task of the current device node before scheduling other device nodes; The finite positive integer c n,m (t) represents the total number of time slots required for server node m to complete the computing task of device node n in the current time slot t; S112: offloading the data packet to a fixed server or a vehicle-mounted mobile server for calculation; in order to better describe the scheduling strategy, O n (t) = {0, 1, 2} to represent the task offloading decision of device node n in time slot t, specifically: n (t) = 0 means that the server node does not schedule the device node n in time slot t; n (t) = 1 means that the device node n is scheduled to the vehicle-mounted mobile server for computing and processing in time slot t; n (t) = 2 means that the device node n is scheduled to the fixed server for computing processing in time slot t; When uninstalling decision O n When (t) = 1, the device node n chooses to offload the computing task to the nearby vehicle-mounted mobile server, and the moving distance of the vehicle is negligible; that is, the uplink transmission rate remains constant during the offloading process, and the total time required to complete the computing task is calculated as: in, represents the total time required for device node n to offload the computing task to the vehicle-mounted mobile server m in time slot t, D n (t) represents the computing task of device node n in time slot t, R n,m (t) represents the uplink transmission rate from device node n to server node m at time slot t, C represents the number of CPU cycles required to process 1 bit of data, represents the calculation frequency of the vehicle-mounted mobile server m; When uninstalling decision O n (t) = 2, the device node n offloads the computing task to the fixed server for computing. Since the device node and the fixed server are fixed in position, the uplink transmission rate will not change, so the total time required to complete the computing task is calculated as: in, represents the total time required for device node n to offload computing tasks to a fixed server in time slot t, Indicates the calculation frequency of a fixed server.

4. The vehicle-mounted edge computing network scheduling method according to claim 3 is characterized in that: In step S1, the information age update model of the system is constructed, which specifically includes the following steps: S121: The information age is the time that has passed since the destination received the latest data packet. Considering that the information age of the buffer will affect the information age of the destination, let g n (t)∈{0,1} indicates whether device node n generates a new task in time slot t. If so, g n (t)=1, otherwise g n (t) = 0; the information age of the data packet in the buffer δ n s (t) is expressed as follows: S122: The information age of the data at the target node is affected by the transmission time and the server calculation time. According to the total time required for the device node n to offload the task to the server node m under different scheduling methods, the rounding up function is used. Discretize the total time into the total number of time slots c n,m (t) is: In addition, define l n (t)∈{0,1,…,c n,m (t)} represents the number of data packets remaining from task n of device node calculated by the server node in time slot t; let i m (t)∈{0,1} represents the state of the server in time slot t, where i m (t) = 1 means server m is in computing state, otherwise i m (t) = 0; l n The updating process of (t) is: Among them, y n,m (t)∈{0,1} represents the state of device node n being scheduled by server m at the beginning of time slot t, y n,m (t) = 1 means that the computing task of device node n is scheduled to server node m, otherwise y n,m (t) = 0; k n,m (t)∈{0,1} indicates whether the device node n successfully sends the data packet to the server node m; S123: When the server node completes the computing task, it transmits the result back to the corresponding device node; represents the information age of the destination device node n, and The update process is expressed as: Among them l n (t) represents the number of data packets remaining from task n of device node calculated by the server node in time slot t, i m (t) represents the state of server m at time slot t.

5. The vehicle-mounted edge computing network scheduling method according to claim 4, characterized in that: Step S2 specifically includes the following steps: S21: Under the condition of long-term time T→∞, the data freshness of the network is measured by the average information age of the device node at the destination. Under any feasible strategy π, the expected average information age of the device node at the destination is It is expressed as: in, Indicates the initialization state information age of the system; S22: Define Π to represent the set of all feasible scheduling strategies, where π∈Π, π * is the optimal scheduling strategy; the optimization problem of minimizing the long-term average information age of the system is expressed as: in, represents the location set of the vehicle-mounted mobile server at time slot t, represents the location set of the device node in time slot t, G m＝0 (t) represents the position of the fixed server at time slot t, R represents the maximum radius at which the fixed server can establish a connection, r represents the maximum radius at which the vehicle-mounted mobile server can establish a connection, ||·|| 2 represents the Euclidean distance function; S23: Establish the optimization problem in step S22 as a Markov decision process; let v n (t)∈{0,1,…,M-1} represents the index of the optimal offloading server node of device node n at time slot t, v n (t) = 0 means that there is no vehicle-mounted mobile server that can be unloaded around device node n in time slot t, and it can only be unloaded to the fixed server; the state space of the system includes the information age of the data packets of all device nodes, the operating status of the server, and the location information of the vehicle-mounted mobile server; therefore, the state space of the system is established as: wherein, v(t) = (v0(t), …, v N-1 (t)), i(t) = (i0(t), …, i M-1 (t)), l(t) = (l0(t), …, l N-1 (t)); S24: The server node in the system selects a device node for scheduling. Let a m (t)∈{-1,0,1,…,N-1} represents the device node index scheduled by server node m at time slot t; where a m (t) = -1 means that the server node m is not currently scheduling any device node, a m (t)≥0 indicates that server node m schedules the index of device node n in time slot t. The combination of actions of all server nodes constitutes the action of the system, which is specifically expressed as: a(t)=(a0(t),…,a M-1 (t)) For simplicity, the matrix ω(t) with dimension N×M is set to represent the link pairs of the server node scheduling device nodes to indicate the scheduling status of the current time slot t, thus obtaining: Among them, y n,m (t)∈{0,1} represents the indicator function of whether the computing task of device node n is scheduled to server node m or not. If y n,m (t) = 1, which means that the computing task of device node n is scheduled to server node m, and a is obtained. m (t) = n; S25: Establish the reward function r(t) of the system as: Among them, r b Represents the benchmark value of the long-term average information age, given by Conclude.

6. The vehicle-mounted edge computing network scheduling method according to claim 5, characterized in that: Step S3 specifically includes the following steps: S31: Introducing the improved Dueling DQN neural network structure with bifurcation structure to optimize the action space, optimizing it from exponential growth to linear growth, specifically including: dividing the network into M bifurcation structures corresponding to the number of server nodes, each bifurcation structure is assigned to a specific server node; the output dimension of each bifurcation structure corresponds to the number of executable sub-actions N+1 on its server; in addition, setting the value function network and advantage function network in the deep network Q network, and the advantage function of the bifurcation sub-action is expressed by H(s,a m |θ) represents, V(s|θ) represents the state estimate of the online network; therefore, the Q value Q of the sub-action in the online network m (s,a|θ) is expressed as: in, represents any executable sub-action of the fork structure, θ t represents the parameters of the online network at time slot t; S32: Introduce action mask to prevent invalid action selection, specifically including: after calculating the Q value of each sub-action, the algorithm verifies whether the device node is within the signal coverage of the server according to the location information of the vehicle-mounted mobile server, and checks whether the current channel status is suitable for unloading. If the device node n does not meet the unloading conditions, assign a small value to the Q value of n in the fork m. Therefore, the Q value of fork m is updated to: Among them, v n (t) represents the optimal unloading object of device node n in time slot t, Q m (s(t),a m (t)|θ t ) represents the Q value of the forked sub-action obtained through the online network.

7. The vehicle-mounted edge computing network scheduling method according to claim 6, characterized in that: Step S4 specifically includes the following steps: S41: After step S3, an improved Dueling DQN network is obtained, which includes an online network and a target network. Both networks include a shared decision module and a deep Q network with a multi-fork structure improvement. The algorithm inputs the system state space. According to the ε-greedy strategy, the agent selects the action combination with the largest Q value with a probability of 1-ε, and randomly selects an action combination from the action space with a probability of ε. After executing the action, (s(t), a(t), r(t), s(t+1)) is stored in the experience replay pool to form an experience. S42: Randomly sample batch size experience in the experience replay pool for training. Then, according to step S31, the online network subtracts the average advantage of all sub-actions in the fork from the advantage of the sub-action and combines it with the state value to obtain the Q value of the sub-action. Since the network structure of the target network Q′ and the online network Q is the same, the forked target network and its parameters are defined as Q' respectively. m (s,a' m |θ') and θ', for each bifurcation, the target value y m (t) is calculated as: Among them, r(t) represents the timely reward, γ represents the discount factor, and a' m is any optional sub-action of forking m in the online network; S43: According to the estimated value and the target value returned by the target value network, the loss function Loss is obtained as: Where M represents the number of forks, y m (t) represents the target value, Q m (s(t),a m (t)|θ t ) represents the Q value of the online network; S44: Based on the loss function of step S43, the gradient loss function ▽Loss is calculated as: in, Represents the estimated network parameter θ t The gradient vector, Q m (s(t),a m (t)|θ t ) represents the Q value of the fork; S45: Every several steps, the network parameters of the estimated network are updated to the target network until the network converges.

Images