Information age-based scheduling method for vehicle-mounted edge computing network

Abstract

The application relates to a vehicle-mounted edge computing network scheduling method based on information age, and belongs to the mobile edge computing field. The method is used for a vehicle-mounted edge computing network in which a fixed server and a vehicle-mounted mobile server coexist, and aims to minimize the long-term average information age of the system by selecting a device node combination scheduled by the server in each time slot. The position constraint, the link conflict constraint and the server computing resource constraint of the vehicle-mounted mobile server are considered, the optimization problem is modeled as a Markov decision process, and a deep duel Q network is improved by using a bifurcation structure and an action mask to obtain a scheduling strategy of the device node. By introducing the bifurcation structure and the action mask into the deep duel Q network, the complexity of problem solving is reduced, and the freshness of data delivery of the vehicle-mounted edge computing network system is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the field of mobile edge computing and relates to a vehicle-mounted edge computing network scheduling method based on information age. Background Art

[0002] The continuous development of edge computing technology has significantly improved data processing and transmission speeds. This not only enhances the ability to process data instantly but also significantly promotes the adoption and development of real-time applications such as smart factories and the Internet of Vehicles. In these computationally intensive and latency-sensitive application scenarios, data freshness is particularly important. Therefore, the Age of Information (AoI) has been proposed as a new data freshness metric. Unlike traditional metrics such as latency, AoI comprehensively considers factors such as transmission time and sending interval, providing a comprehensive measure of the freshness of data received at the destination.

[0003] In vehicular edge computing scenarios, vehicles such as smart mining trucks and smart logistics vehicles are typically equipped with abundant computing resources. These devices not only efficiently process their own data and tasks but also provide computing services to surrounding nodes with limited computing power during idle time. In an IoT system assisted by vehicular edge computing, fixed servers deployed at base stations and mobile servers onboard vehicles jointly provide computing services to devices in the network. Through rational task scheduling and resource allocation, the system's information age is minimized to ensure data freshness. However, due to the varying computing power and transmission distances between fixed and mobile servers, dynamically selecting a server to handle computing tasks presents a challenge. Furthermore, the dynamic mobility of mobile servers further complicates scheduling strategy design. Therefore, developing a scheduling method for device nodes that minimizes the system's long-term average information age and improves information freshness, while satisfying constraints such as channel availability, computing resources, and the location of mobile servers, presents a significant challenge. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a vehicle-mounted edge computing network scheduling method based on information age. For the scenario where fixed servers and vehicle-mounted mobile servers exist in the vehicle-mounted edge computing network system, the computing resources of the edge computing server are comprehensively considered, and the computing tasks of the device nodes are scheduled to the edge computing server for processing. The present invention constructs a vehicle-mounted edge computing-assisted computing network information age update model, and expresses the optimization problem of minimizing the long-term average information age of the system as a Markov decision process. In order to solve this optimization problem, the present invention further introduces a bifurcation structure and an action mask, thereby improving the traditional deep reinforcement learning algorithm. This improvement not only reduces the state space and action space in the deep reinforcement learning method, but also speeds up the learning speed of the scheduling method, thereby improving the data freshness of the system.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A scheduling method for vehicular edge computing networks based on information age is proposed. In a computing network system model where fixed servers and vehicular mobile servers coexist, a deep reinforcement learning scheduling algorithm improved with bifurcating structures and action masks is used to optimize the scheduling strategy in a system model where both servers provide computing services. Specifically, the computing tasks of device nodes are scheduled to idle server nodes for processing in each time slot. This method minimizes the long-term average information age of the system while satisfying the resource constraints of the edge computing servers, link conflict constraints, and the location constraints of the vehicular mobile servers.

[0007] The method specifically comprises the following steps:

[0008] S1: Obtain relevant parameter information of the vehicle-mounted edge computing network system, build the system's communication model, task scheduling model, and information age update model, and establish a scheduling network for learning the optimal scheduling strategy;

[0009] S2: Construct an optimization problem to minimize the system's long-term average information age. Establish the system's state space, action space, and reward function based on the communication model, task scheduling model, and information age update model. This transforms the strongly coupled optimization problem of server computing resources, link conflicts, and the location of the onboard mobile server into a Markov decision process.

[0010] S3: Updates the system state at each time slot, optimizes the action space of the neural network through a shared decision module and bifurcating structure, and introduces an action mask mechanism to mask the Q value of an action in a specific state, thereby further reducing the number of optional action spaces and improving the convergence speed and robustness of the algorithm.

[0011] S4: Train the scheduling network, calculate the loss function, and then update the network parameters through the gradient descent algorithm until the network converges. Finally, a link scheduling strategy suitable for the vehicle-mounted mobile server-assisted IoT device system is obtained.

[0012] Furthermore, in step S1, a communication model of the system is constructed, which specifically includes the following steps:

[0013] S101: Consider an IoT system consisting of M server nodes, including M-1 vehicle-mounted mobile servers and one fixed server, which are used to provide computing services for N device nodes. The computing tasks generated by the device nodes are transmitted to the server nodes through wireless channels for processing, and the computing results are returned to the corresponding device nodes. Compared with fixed servers, vehicle-mounted mobile servers are mobile and can be closer to device nodes, thereby improving the transmission quality of the wireless channel. Let the server index be Where m = 0 represents a fixed server; in addition, a separate definition of the vehicle-mounted mobile server set is Likewise, the index of a device node is defined as The location of the device node is fixed; the system processing task is based on the time slot. represents the index of the time slot. At the beginning of each time slot, the system can schedule at most min{M,N} device nodes to transmit data packets through the orthogonal channel. Considering the distortion and thermal noise in the wireless channel, we assume that data transmission is not always successful. After the data packet is successfully transmitted, it will be calculated and processed by the corresponding server.

[0014] S102: Set the fixed server as the coordinate origin to establish a coordinate system, whose coordinates are G m (t)=(0,0), the device node coordinates can be established as The coordinates of the vehicle-mounted mobile server can be expressed as At time slot t, the Euclidean distance d between device node n and server node m n,m (t) is:

[0015]

[0016] in, and (x m (t),y m (t)) represent the location coordinates of device node n and server node m respectively;

[0017] S103: Wireless channel gain h between device node n and server node m n,m (t) is:

[0018] h n,m (t)=h0(d n,m (t))-2

[0019] Where h0 is the unit power gain of the channel;

[0020] S104: According to Shannon's formula, the uplink transmission rate R from device node n to server node m n,m (t) is:

[0021]

[0022] in, is the Gaussian noise power, B is the bandwidth, is the transmission power of device node n, h n,m (t) is the wireless channel gain from device node n to server node m.

[0023] Furthermore, in step S1, a task scheduling model of the system is constructed, which specifically includes the following steps:

[0024] S111: Assume that each computing task can only be scheduled by one server node, and that a single server node cannot schedule computing tasks for multiple device nodes in the same time slot. Since the amount of computing results is generally small, the transmission time from the server node back to the device node can be ignored. The device node generates computing tasks (composed of data packets) according to the parameter λ. n ∈(0,1] Bernoulli distribution; each device node uses a buffer to store data packets. When a new data packet is generated, the old data packet in the buffer will be replaced; the server node must complete the calculation task of the current device node before scheduling other device nodes; a finite positive integer c n,m (t) represents the total number of time slots required for server node m to complete the computation task of device node n in the current time slot t;

[0025] S112: The present invention considers two types of task offloadable objects, namely offloading data packets to fixed servers or vehicle-mounted mobile servers for calculation; in order to better describe the scheduling strategy, O n (t) = {0, 1, 2} represents the task offloading decision of device node n in time slot t, specifically: n (t) = 0 means that the server node does not schedule the device node n in time slot t; n (t) = 1 means that the device node n is scheduled to the vehicle-mounted mobile server for computing and processing in time slot t; n (t) = 2 means that the device node n is scheduled to the fixed server for computing processing in time slot t;

[0026] When uninstalling decision O nWhen (t) = 1, the device node n chooses to offload the computing task to the nearby vehicle-mounted mobile server. Since the computing task offloading time in the actual system is usually short, the moving distance of the vehicle can be ignored; that is, the uplink transmission rate remains constant during the offloading process. The total time required to complete the computing task is calculated as:

[0027]

[0028] in, represents the total time required for device node n to offload the computing task to the vehicle-mounted mobile server m in time slot t, D n (t) represents the computing task of device node n in time slot t, R n,m (t) represents the uplink transmission rate from device node n to server node m at time slot t, C represents the number of CPU cycles required to process 1 bit of data, represents the calculation frequency of the vehicle-mounted mobile server m;

[0029] When uninstalling decision O n (t) = 2, the device node n offloads the computing task to the fixed server for computing. Since the device node and the fixed server are fixed in position, the uplink transmission rate does not change. Therefore, the total time required to complete the computing task is calculated as:

[0030]

[0031] in, represents the total time required for device node n to offload the computing task to a fixed server in time slot t, Indicates the calculation frequency of a fixed server.

[0032] Furthermore, in step S1, a system information age update model is constructed, which specifically includes the following steps:

[0033] S121: Information age is the time that has passed since the destination received the latest data packet. Considering that the information age of the buffer will affect the information age of the destination, let g n (t)∈{0,1} indicates whether the device node n generates a new task in time slot t. If so, g n (t)=1, otherwise g n (t) = 0; the information age of the data packet in the buffer δ n The expression of s(t) is as follows:

[0034]

[0035] S122: The information age of the data at the target node is affected by the transmission time and the server calculation time. According to the total time required for the device node n to offload the task to the server node m under different scheduling methods, the rounding function is used. Discretize the total time into the total number of time slots c n,m (t) is:

[0036]

[0037] In addition, define l n (t)∈{0,1,…,c n,m (t)} represents the number of data packets remaining from task n of device node calculated by the server node in time slot t; let i m (t)∈{0,1} represents the state of the server in time slot t, where i m (t) = 1 means that server m is in computing state, otherwise i m (t) = 0; l n The updating process of (t) is:

[0038]

[0039] Among them, y n,m (t)∈{0,1} represents the state of device node n being scheduled by server m at the beginning of time slot t, y n,m (t) = 1 means that the computing task of device node n is scheduled to server node m, otherwise y n,m (t) = 0; k n,m (t)∈{0,1} indicates whether the device node n successfully sends the data packet to the server node m;

[0040] S123: When the server node completes the computing task, it transmits the result back to the corresponding device node; represents the information age of the destination device node n, and The update process is expressed as:

[0041]

[0042] where l n (t) represents the number of data packets remaining from the device node n task calculated by the server node in time slot t, i m (t) represents the state of server m at time slot t.

[0043] Furthermore, step S2 specifically includes the following steps:

[0044] S21: Under the condition of long-term time T→∞, the data freshness of the network is measured by the average information age of the device node at the destination. Under any feasible strategy π, the expected average information age of the device node at the destination is

[0045] Expressed as:

[0046]

[0047] in, Indicates the initialization state information age of the system;

[0048] S22: Define π to represent the set of all feasible scheduling strategies, where π∈π, π * is the optimal scheduling strategy; the optimization problem of minimizing the long-term average information age of the system is expressed as:

[0049]

[0050] in, represents the location set of the vehicle-mounted mobile server at time slot t, represents the location set of the device node in time slot t, G m＝0 (t) represents the position of the fixed server at time slot t, R represents the maximum radius to which the fixed server can establish a connection, r represents the maximum radius to which the vehicle-mounted mobile server can establish a connection, ||·|| 2 represents the Euclidean distance function;

[0051] S23: Establish the optimization problem in step S22 as a Markov decision process; let v n (t)∈{0,1,…,M-1} represents the index of the optimal offloading server node of device node n at time slot t, v n (t) = 0 means that there is no vehicle-mounted mobile server available for offloading around device node n in time slot r, and the offloading can only be done to a fixed server. The state space of the system includes the information age of the data packets of all device nodes, the operating status of the server, and the location information of the vehicle-mounted mobile server. Therefore, the state space of the system is established as:

[0052]

[0053] in, v(t)=(v0(t),…,v N-1 (t)), i(t)=(i0(t),…,i M-1 (t)), l(t)=(l0(t),…,l N-1 (t));

[0054] S24: The server node in the system can select a device node for scheduling, let am (t)∈{-1,0,1,…,N-1} represents the device node index scheduled by server node m at time slot t; where a m (t) = -1 means that the server node m is not currently scheduling any device node, a m (t)≥0 indicates the index of the device node n scheduled by the server node m in time slot t. The combination of actions of all server nodes constitutes the action of the system, which can be specifically expressed as:

[0055] a(t)=(a0(t),…,a M-1 (t))

[0056] For simplicity, we assume that the matrix ω(t) with dimension N×M represents the link pairs of the server node scheduling the device nodes to indicate the scheduling status of the current time slot t, thus obtaining:

[0057]

[0058] Among them, y n,m (t)∈{0,1} represents the indicator function of whether the computing task of device node n is scheduled to server node m or not. If y n,m (t) = 1, which means that the computing task of device node n is scheduled to server node m, and we can get a m (t) = n;

[0059] S25: Establish the reward function r(t) of the system:

[0060]

[0061] Among them, r b The benchmark value of the long-term average information age can be expressed by Come to.

[0062] Furthermore, step S3 specifically includes the following steps:

[0063] S31: The neural network structure of Dueling DQN with the improved bifurcation structure optimizes the action space, optimizing it from exponential growth to linear growth. Specifically, the network is divided into M bifurcation structures corresponding to the number of server nodes, and each bifurcation structure is assigned to a specific server node; the output dimension of each bifurcation structure corresponds to the number of sub-actions that can be executed by its server N+1; in addition, a value function network and an advantage function network are set in the deep network Q network respectively, and the advantage function of the bifurcation sub-action is expressed as H(s,a m |θ) represents, V(s|θ) represents the state estimate of the online network; therefore, the Q value Q of the sub-action in the online network is m (s,a|θ) can be expressed as:

[0064] Q m (s(t),a m (t)|θ t )=V(s(t)|θ t )+H(s(t),a m (t)|θ t )

[0065]

[0066] in, represents any executable sub-action of the fork structure, θ t represents the parameters of the online network at time slot t;

[0067] S32: Further introduce action mask to prevent invalid action selection, specifically including: after calculating the Q value of each sub-action, the algorithm verifies whether the device node is within the signal coverage of the server based on the location information of the vehicle mobile server, and checks whether the current channel status is suitable for unloading. If the device node n does not meet the unloading conditions, the Q value of n in the fork m is assigned a smaller value. Therefore, the Q value of fork m can be updated as:

[0068]

[0069] Among them, v n (t) represents the optimal unloading object of device node n in time slot t, Q m (s(t),a m (t)|θ t ) represents the Q value of the forked sub-action obtained through the online network.

[0070] Further, step S4 specifically includes the following steps:

[0071] S41: After step S3, the improved Dueling DQN network is obtained, which includes an online network and a target network. Both networks include a shared decision module and a deep Q network with an improved multi-fork structure. The algorithm inputs the system state space. According to the ε-greedy strategy, the agent selects the action combination with the largest Q value with a probability of 1-ε, and randomly selects an action combination from the action space with a probability of ε. After executing the action, (s(t), a(t), r(t), s(t+1)) is stored in the experience replay pool to form an experience.

[0072] S42: Randomly sample batch size experience from the experience replay pool for training. Then, according to step S31, the online network subtracts the average advantage of all sub-actions in the bifurcation from the advantage of the sub-action and combines it with the state value to obtain the Q value of the sub-action. Since the network structure of the target network Q' and the online network Q is the same, the bifurcation target network and its parameters are defined as Q' respectively. m (s,a' m |θ') and θ', for each bifurcation, the target value y m (t) can be calculated as:

[0073]

[0074] Among them, r(t) represents the timely reward, γ represents the discount factor, and a' m is any optional sub-action of forking m in the online network;

[0075] S43: According to the estimated value and the target value returned by the target value network, the loss function Loss is obtained as:

[0076]

[0077] Where M represents the number of forks, y m (t) represents the target value, Q m (s(t),a m (t)|θ t ) represents the Q value of the online network;

[0078] S44: Based on the loss function of step S43, calculate the gradient loss function for:

[0079]

[0080] in, Represents the estimated network parameters θ t The gradient vector, Q m (s(t),a m (t)|θ t ) represents the Q value of the bifurcation;

[0081] S45: Every several steps, the network parameters of the estimated network are updated to the target network until the network converges.

[0082] The beneficial effects of the present invention are:

[0083] (1) Based on comprehensive consideration of constraints such as edge server computing resources, link conflicts, and the location of onboard mobile servers, the present invention optimizes the long-term average information age of the system by training an autonomous learning scheduling method, thereby improving the freshness of system data delivery.

[0084] (2) This paper improves the Deep Duel Q Network by introducing a multi-fork structure, reducing the state space and action space in the network representation. In addition, the adoption of an action mask mechanism enhances the stability of the training process, thereby significantly improving the learning speed of the scheduling method. This enables the scheduling method to be efficiently deployed in an onboard edge computing network.

[0085] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0087] Figure 1 This is a schematic diagram of the structure of the computing network scheduling system assisted by vehicle-mounted edge computing based on information age of the present invention;

[0088] Figure 2 Schematic diagram of the deep duel Q network and action mask mechanism with improved bifurcation structure adopted by the present invention;

[0089] Figure 3 This is a flow chart of the vehicle-mounted edge computing network scheduling method based on information age of the present invention. DETAILED DESCRIPTION

[0090] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0091] See also Figures 1 to 3The present invention aims at a computing network system assisted by an on-board edge computing server, comprehensively considering constraints such as server computing resources, link conflicts, and the location of on-board mobile servers, to minimize the long-term average information age of the system. A deep reinforcement learning method with an improved bifurcation structure is adopted to split the large action space originally caused by multiple edge server nodes into bifurcated small action spaces divided according to the number of server nodes, thereby reducing the dimension of the action space, accelerating the algorithm training speed, and obtaining a scheduling method for a computing network system assisted by an on-board edge computing server. The present invention further introduces an action mask mechanism to enhance the stability of the training process and accelerate the learning speed of the scheduling method.

[0092] Figure 1 Schematic diagram of the computing network system structure assisted by the vehicle-mounted edge computing server. The system is an IoT system that provides computing services for N device nodes by M server nodes (including M-1 vehicle-mounted mobile servers and one fixed server). The computing tasks generated by the device nodes are transmitted to the server nodes through wireless channels for processing, and the calculation results are returned to the corresponding device nodes. Compared with fixed servers, vehicle-mounted mobile servers can be closer to device nodes, thereby improving the quality of wireless channels. The server index is denoted as Where m=0 represents a fixed server. In addition, a separate definition of the vehicle-mounted mobile server set is defined as Likewise, the index of a device node is defined as And the location of the device node is fixed.

[0093] The system processes computing tasks based on time slots, and Represents the index of a time slot. At the beginning of each time slot, the system can schedule data packets from up to min{M,N} device nodes for transmission via orthogonal channels. Given the distortion and thermal noise in wireless channels, we assume that data transmission is not always successful. After successful transmission, data packets are processed on the corresponding server. Since the amount of computational results is generally small, the transmission time from the server node to the device node is negligible.

[0094] Figure 2 This is a schematic diagram of the improved bifurcation structure Q network and action mask mechanism used in the present invention. It mainly consists of a shared decision module, a multi-bifurcation structure, and an action mask. The network is divided into M bifurcation structures, each of which is assigned to a specific server node. The output dimension of each bifurcation structure corresponds to the number of executable sub-actions N+1 of the server node. Let represents any executable sub-action of a fork structure, a m =-1 means the server node does not schedule, a m=n represents the device node n scheduled by server node m. The Q value of a sub-action can be obtained by adding the advantage value of the sub-action to the state value and then subtracting the average advantage of all sub-actions in the fork:

[0095]

[0096] Among them, H(s,a m |θ) represents the advantage value of the sub-action, and V(s|θ) represents the state estimation value of the online network.

[0097] After calculating the Q value of each sub-action, verify whether the device node is within the signal coverage of the server based on the location information of the vehicle mobile server, and check whether the current channel status is suitable for unloading. If the device node n does not meet the unloading conditions, assign a smaller value to the Q value of n in the fork m. Therefore, the Q value of fork m can be updated as:

[0098]

[0099] Figure 3 This is a flow chart of the computing network scheduling method assisted by vehicle-mounted edge computing based on information age of the present invention, as shown in FIG. Figure 3 As shown, the specific steps include:

[0100] V1-V4: Obtain the computing network system model parameters assisted by the on-board edge computing server, initialize the parameters in the network, and determine the network's state space, action space, and reward function.

[0101] V5~V6: Obtain the location of the vehicle-mounted mobile server, calculate the theoretical time slot for each device node to complete the calculation under different offloading objects, and update the system state space.

[0102] V7~V11: Use the ε-greedy action exploration strategy. If the generated ε is greater than the set threshold ε greedy , then randomly select an action output in the action space, otherwise select the action with the largest Q value as the output, and execute the scheduling decision in the current state, calculate the reward function, and store the current system state, action, reward and next state as experience in the experience replay pool.

[0103] V12~V16: Randomly select experience sets from the experience replay pool to form training segments, calculate the estimated value and target value of each bifurcation, and assign an action mask to the Q value through the action mask mechanism. Then, calculate the loss function using the estimated value and target value, calculate the gradient and update the network weights. Every few steps, update the parameters of the estimated network to the target network.

[0104] V17~V19: After determining whether the loss function is stable and the training termination conditions are met, if the termination conditions are not met, execute V5. If they are met, link scheduling offloading is performed according to the fully trained scheduling network.

[0105] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A vehicle-mounted edge computing network scheduling method based on information age, characterized in that: In a computing network system model where fixed servers and on-board mobile servers coexist, a deep reinforcement learning scheduling algorithm improved by bifurcating structure and action mask is used to optimize its scheduling strategy in a system model where both servers provide computing services. Specifically, the computing tasks of device nodes are dispatched to idle server nodes for processing in each time slot. This minimizes the long-term average information age of the system while satisfying the resource constraints of the edge computing servers, link conflict constraints, and the location constraints of the on-board mobile servers. The method specifically comprises the following steps: S1: Obtain relevant parameter information of the vehicle-mounted edge computing network system, build the system's communication model, task scheduling model, and information age update model, and establish a scheduling network for learning the optimal scheduling strategy; S2: Construct an optimization problem to minimize the system's long-term average information age. Establish the system's state space, action space, and reward function based on the communication model, task scheduling model, and information age update model. This transforms the strongly coupled optimization problem of server computing resources, link conflicts, and the location of the onboard mobile server into a Markov decision process. S3: Update the system state in each time slot, optimize the action space of the neural network through a shared decision module and bifurcation structure, and introduce an action mask mechanism to assign a mask to the action Q value in a specific state; Step S3 specifically includes the following steps: S31: The neural network structure of Dueling DQN with improved bifurcation structure is introduced to optimize the action space, which is optimized from exponential growth to linear growth. Specifically, the network is divided into M bifurcation structures corresponding to the number of server nodes, and each bifurcation structure is assigned to a specific server node; the output dimension of each bifurcation structure corresponds to the number of sub-actions that can be executed by its server N+1; in addition, a value function network and an advantage function network are set in the deep network Q network respectively, and the advantage function of the bifurcation sub-action is expressed as H(s,a m |θ) represents, V(s|θ) represents the state estimate of the online network; therefore, the Q value Q of the sub-action in the online network is m (s,a|θ) is expressed as: in, represents any executable sub-action of the fork structure, θ t represents the parameters of the online network in time slot t, and N represents the number of device nodes; S32: Introduce action mask to prevent invalid action selection, specifically including: after calculating the Q value of each sub-action, the algorithm verifies whether the device node is within the signal coverage of the server based on the location information of the vehicle mobile server, and checks whether the current channel status is suitable for unloading. If the device node n does not meet the unloading conditions, the Q value of n in the fork m is assigned a smaller value. Therefore, the Q value of fork m is updated to: Among them, v n (t) represents the optimal unloading object of device node n in time slot t, Q m (s(t),a m (t)|θ t ) represents the Q value of the forked sub-action obtained through the online network; S4: Train the scheduling network, calculate the loss function, and then update the network parameters through the gradient descent algorithm until the network converges. Finally, a link scheduling strategy suitable for the vehicle-mounted mobile server-assisted IoT device system is obtained.

2. The vehicle-mounted edge computing network scheduling method according to claim 1, characterized in that: In step S1, a communication model of the system is constructed, which specifically includes the following steps: S101: Consider an IoT system consisting of M server nodes, including M-1 vehicle-mounted mobile servers and one fixed server, which provide computing services for N device nodes. The computing tasks generated by the device nodes are transmitted to the server nodes through wireless channels for processing, and the computing results are returned to the corresponding device nodes. Compared with fixed servers, vehicle-mounted mobile servers are mobile and can be closer to device nodes, thereby improving the transmission quality of the wireless channel. The server index is denoted as Where m = 0 represents a fixed server; in addition, a separate definition of the vehicle-mounted mobile server set is Likewise, the index of a device node is defined as The location of the device node is fixed; the system processing task is based on the time slot. Represents the index of the time slot; at the beginning of each time slot, the system can schedule at most min{M,N} device nodes to transmit data packets through the orthogonal channel. Considering the distortion and thermal noise in the wireless channel, it is assumed that data transmission is not always successful. After the data packet is successfully transmitted, it will be calculated and processed on the corresponding server; S102: Set the fixed server as the coordinate origin to establish a coordinate system, whose coordinates are G m (t)=(0,0), the device node coordinates are established as The coordinates of the vehicle-mounted mobile server are expressed as At time slot t, the Euclidean distance d between device node n and server node m n,m (t) is: in, and (x m (t),y m (t)) represent the location coordinates of device node n and server node m respectively; S103: Wireless channel gain h between device node n and server node m n,m (t) is: h n,m (t)=h0(d n,m (t)) -2 Where h0 is the unit power gain of the channel; S104: According to Shannon's formula, the uplink transmission rate R from device node n to server node m n,m (t) is: in, is the Gaussian noise power, B is the bandwidth, is the transmission power of device node n, h n,m (t) is the wireless channel gain from device node n to server node m.

3. The vehicle-mounted edge computing network scheduling method according to claim 2, characterized in that: In step S1, a task scheduling model of the system is constructed, which specifically includes the following steps: S111: Assume that each computing task can only be scheduled by one server node, and that a single server node cannot schedule computing tasks for multiple device nodes in the same time slot; the transmission time from the server node back to the device node is negligible; the computing task composed of data packets generated by the device node follows the parameter λ n ∈(0,1] Bernoulli distribution; each device node uses a buffer to store data packets. When a new data packet is generated, the old data packet in the buffer will be replaced; the server node must complete the calculation task of the current device node before scheduling other device nodes; a finite positive integer c n,m (t) represents the total number of time slots required for server node m to complete the computation task of device node n in the current time slot t; S112: Unload the data packet to the fixed server or the vehicle-mounted mobile server for calculation; in order to better describe the scheduling strategy, use O n (t) = {0, 1, 2} represents the task offloading decision of device node n in time slot t, specifically: n (t) = 0 means that the server node does not schedule the device node n in time slot t; n (t) = 1 means that the device node n is scheduled to the vehicle-mounted mobile server for computing and processing in time slot t; n (t) = 2 means that the device node n is scheduled to the fixed server for computing processing in time slot t; When uninstalling decision O n When (t) = 1, the device node n chooses to offload the computing task to the nearby vehicle-mounted mobile server, and the vehicle's moving distance is negligible. That is, the uplink transmission rate remains constant during the offloading process, and the total time required to complete the computing task is calculated as: in, represents the total time required for device node n to offload the computing task to the vehicle-mounted mobile server m in time slot t, D n (t) represents the computing task of device node n in time slot t, R n,m (t) represents the uplink transmission rate from device node n to server node m at time slot t, C represents the number of CPU cycles required to process 1 bit of data, represents the calculation frequency of the vehicle-mounted mobile server m; When uninstalling decision O n (t) = 2, the device node n offloads the computing task to the fixed server for computing. Since the device node and the fixed server are fixed in position, the uplink transmission rate does not change. Therefore, the total time required to complete the computing task is calculated as: in, represents the total time required for device node n to offload the computing task to a fixed server in time slot t, Indicates the calculation frequency of a fixed server.

4. The vehicle-mounted edge computing network scheduling method according to claim 3, characterized in that: In step S1, the system information age update model is constructed, which specifically includes the following steps: S121: Information age is the time that has passed since the destination received the latest data packet. Considering that the information age of the buffer will affect the information age of the destination, let g n (t)∈{0,1} indicates whether the device node n generates a new task in time slot t. If so, g n (t)=1, otherwise g n (t) = 0; the information age of the data packet in the buffer The expression is as follows: S122: The information age of the data at the target node is affected by the transmission time and the server calculation time. According to the total time required for the device node n to offload the task to the server node m under different scheduling methods, the rounding function is used. Discretize the total time into the total number of time slots c n,m (t) is: In addition, define l n (t)∈{0,1,…,c n,m (t)} represents the number of data packets remaining from task n of device node calculated by the server node in time slot t; let i m (t)∈{0,1} represents the state of the server in time slot t, where i m (t) = 1 means that server m is in computing state, otherwise i m (t) = 0; l n The updating process of (t) is: Among them, y n,m (t)∈{0,1} represents the state of device node n being scheduled by server m at the beginning of time slot t, y n,m (t) = 1 means that the computing task of device node n is scheduled to server node m, otherwise y n,m (t) = 0; k n,m (t)∈{0,1} indicates whether the device node n successfully sends the data packet to the server node m; S123: When the server node completes the computing task, it transmits the result back to the corresponding device node; represents the information age of the destination device node n, and The update process is expressed as: where l n (t) represents the number of data packets remaining from the device node n task calculated by the server node in time slot t, i m (t) represents the state of server m at time slot t.

5. The vehicle-mounted edge computing network scheduling method according to claim 4, characterized in that: Step S2 specifically includes the following steps: S21: Under the condition of long-term time T→∞, the data freshness of the network is measured by the average information age of the device node at the destination. Under any feasible strategy π, the expected average information age of the device node at the destination is Expressed as: in, Indicates the initialization state information age of the system; S22: Define π to represent the set of all feasible scheduling strategies, where π∈π, π * is the optimal scheduling strategy; the optimization problem of minimizing the long-term average information age of the system is expressed as: O n (t)∈{0,1,2}, in, represents the location set of the vehicle-mounted mobile server at time slot t, represents the location set of the device node in time slot t, G m＝0 (t) represents the position of the fixed server at time slot t, R represents the maximum radius to which the fixed server can establish a connection, r represents the maximum radius to which the vehicle-mounted mobile server can establish a connection, ||·|| 2 represents the Euclidean distance function; S23: Establish the optimization problem in step S22 as a Markov decision process; let v n (t)∈{0,1,…,M-1} represents the index of the optimal offloading server node of device node n at time slot t, v n (t) = 0 means that there is no vehicle-mounted mobile server available for offloading around device node n at time slot t, and the offloading can only be done to a fixed server. The system state space includes the information age of the data packets of all device nodes, the operating status of the server, and the location information of the vehicle-mounted mobile server. Therefore, the state space of the system is established as: where, v(t) = (v0(t), …, v N-1 (t)), i(t) = (i0(t), …, i M-1 (t)), l(t) = (l0(t), …, l N-1 (t)); S24: The server node in the system selects a device node for scheduling, let a m (t)∈{-1,0,1,…,N-1} represents the device node index scheduled by server node m at time slot t; where a m (t) = -1 means that the server node m is not currently scheduling any device node, a m (t)≥0 indicates the index of the device node n scheduled by the server node m in time slot t. The combination of actions of all server nodes constitutes the action of the system, which can be expressed as: a(t)=(a0(t),…,a M-1 (t)) For simplicity, we assume that the matrix ω(t) with dimension N×M represents the link pairs of the server node scheduling the device nodes to indicate the scheduling status of the current time slot t, thus obtaining: Among them, y n,m (t)∈{0,1} represents the indicator function of whether the computing task of device node n is scheduled to server node m or not. If y n,m (t) = 1, which means that the computing task of device node n is scheduled to server node m, and a is obtained. m (t) = n; S25: Establish the reward function r(t) of the system as: Among them, r b The benchmark value of the long-term average information age is represented by Come to.

6. The vehicle-mounted edge computing network scheduling method according to claim 5, characterized in that: Step S4 specifically includes the following steps: S41: After step S3, the improved Dueling DQN network is obtained, which includes an online network and a target network. Both networks include a shared decision module and a deep Q network with an improved multi-fork structure. The algorithm inputs the system state space. According to the ε-greedy strategy, the agent selects the action combination with the largest Q value with a probability of 1-ε, and randomly selects an action combination from the action space with a probability of ε. After executing the action, (s(t), a(t), r(t), s(t+1)) is stored in the experience replay pool to form an experience. S42: Randomly sample batch size experience from the experience replay pool for training. Then, according to step S31, the online network subtracts the average advantage of all sub-actions in the fork from the advantage of the sub-action and combines it with the state value to obtain the Q value of the sub-action. Since the network structure of the target network Q' and the online network Q is the same, the target network and its parameters are defined as Q' respectively. m (s,a' m |θ') and θ', for each bifurcation, the target value y m (t) is calculated as: Among them, r(t) represents the timely reward, γ represents the discount factor, and a' m is any optional sub-action of forking m in the online network; S43: According to the estimated value and the target value returned by the target value network, the loss function Loss is obtained as: Where M represents the number of forks, y m (t) represents the target value, Q m (s(t),a m (t)|θ t ) represents the Q value of the online network; S44: Based on the loss function of step S43, calculate the gradient loss function for: in, Represents the estimated network parameters θ t The gradient vector, Q m (s(t),a m (t)|θ t ) represents the Q value of the bifurcation; S45: Every several steps, the network parameters of the estimated network are updated to the target network until the network converges.

Images