Internet of vehicles double time scale network slice resource allocation method based on digital twinning

Abstract

The application relates to a kind of vehicle networking double time scale network slice resource allocation methods based on digital twinning, belong to mobile communication technical field, construct vehicle networking network slice architecture based on digital twinning, realize real-time monitoring of vehicle and network resource, in order to guarantee the fidelity of digital twinning network, it is divided into fidelity of perception process, fidelity of data transmission process and fidelity of information age guarantee three parts;Under the condition of guaranteeing vehicle QoS, the optimization model with the maximum digital twinning fidelity and system spectral efficiency as the target is established, it is decomposed into two sub-problems in time scale, and the TCN-DDPG method is proposed;In large time scale, TCN is used to predict the resource demand of network slice, and resource allocation between slices is realized;In small time scale, DDPG is used to schedule resource blocks to vehicle users.

Description

Technical Field

[0001] The present invention belongs to the field of mobile communication technology and relates to a method for allocating dual-time-scale network slice resources for Internet of Vehicles based on digital twins. Background Art

[0002] The Internet of Vehicles (IoV) refers to technologies and systems that connect vehicles to external networks and services via the internet. By leveraging advanced sensing and networking technologies to comprehensively perceive and control vehicle traffic on the road, IoV is expected to provide a variety of services to ensure safe and efficient driving, paving the way for autonomous driving. At the same time, as the number of services and applications in vehicles increases, network technology needs to flexibly and intelligently allocate resources to meet IoV needs. Therefore, academic research has explored network slicing (NS) and artificial intelligence (AI)-powered communication network technologies to adapt to these applications and services. Network slicing is primarily categorized as core network (CN) slicing and radio access network (RAN) slicing. However, RAN slicing still presents significant challenges due to the random nature of RANs, the multi-dimensional quality of service (QoS) requirements of services, and the limited network terminal resources.

[0003] DT technology uses modeling, computing, and communication to create a virtual twin of the physical network, enabling synchronization of virtual and physical spaces and data processing. Combining network slicing with digital twin technology for connected vehicles enables real-time vehicle monitoring and intelligent management and optimization of network resources. By creating digital twin models of vehicles and network elements, vehicle behavior can be simulated and vehicle status monitored in real time. Network resources, including bandwidth, computing resources, and storage resources, can also be virtually modeled and simulated. Based on the analysis results of the digital twin model, network slicing can dynamically adjust and allocate network resources to meet the needs of different slices and applications, improving network resource utilization efficiency and performance.

[0004] While existing technologies have explored aspects such as digital twin construction, information age, and resource allocation, they lack consideration of the fidelity of digital twin construction. Most approaches assume high-fidelity digital twins. Low-fidelity digital twin networks exhibit significant deviations from physical networks, leading to delays and errors in monitoring vehicle status and slice resources. This can cause resource allocation policies in digital twin functional models to lag or even err, impacting vehicle users' QoS. Furthermore, limited research has been conducted on resource allocation in scenarios where digital twins are combined with IoV network slicing. In these scenarios, digital twin networks can generate future states of vehicles and network elements and verify policies against these future states. High-fidelity digital twin networks generate highly reliable future states and policies, significantly reducing verification time for future slice resource allocation policies and enabling direct deployment within the physical network. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a dual-time-scale network slice resource allocation method for Internet of Vehicles based on digital twins.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for allocating network slice resources at dual time scales in an Internet of Vehicles (IoV) based on digital twins includes the following steps:

[0008] S1: Build a digital twin-based IoV network slicing architecture. By building digital twins of physical entities in digital twin servers, real-time monitoring of vehicles, base stations, and network resources is achieved.

[0009] S2: Divide the digital twin network into three parts: the fidelity of the perception process, the fidelity of the data transmission process, and the fidelity of the information age guarantee, in order to establish a high-fidelity digital twin network;

[0010] S3: Under the condition of ensuring vehicle service quality (QoS), an optimization model is established with the goal of maximizing the fidelity of digital twins and system spectrum efficiency. It is decomposed into two sub-problems on the time scale, and a deep deterministic policy gradient (TCN-DDPG) resource allocation method based on temporal convolutional network is proposed: in large time scales, temporal convolutional network (TCN) is used to predict the resource requirements of network slices and realize resource allocation between slices; in small time scales, deep deterministic policy gradient (DDPG) is used to schedule resource blocks to vehicle users, which is used to decide on a resource block allocation method that satisfies various constraints and maximizes the fidelity of digital twins and system spectrum efficiency.

[0011] Furthermore, in step S1: the Internet of Vehicles network slicing architecture based on digital twins includes a physical layer, a digital twin layer, and an application layer; the physical layer includes vehicles, base stations, and a central controller; the digital twin layer includes a data warehouse, a basic model, and a functional model; the application layer includes various vehicle applications, wherein:

[0012] The data warehouse is used to collect and store real-time data of physical entities;

[0013] The basic model establishes a digital twin from the data stored in the data warehouse, thereby constructing a digital twin network of the Internet of Vehicles network slice;

[0014] The functional model is used to form a closed loop with the basic model to perform strategy verification.

[0015] Furthermore, the step S2 specifically includes:

[0016] The fidelity of the perception process is measured by perception accuracy. The greater the perception frequency, the better the perception accuracy, while the perception energy consumption increases. The optimal perception accuracy and the minimum perception energy consumption solution are obtained by combining the perception accuracy and the perception energy consumption.

[0017] The fidelity of the data transmission process is measured by the signal-to-noise ratio during the transmission process. If the signal-to-noise ratio is less than a threshold, the information will be erroneous.

[0018] The fidelity of the information age guarantee is measured by the information age of the transmitted information. Maximizing the fidelity of the information age guarantee means minimizing the information age of the transmitted information.

[0019] Furthermore, the optimization model described in step S3 with the goal of maximizing the fidelity of the digital twin and the system spectrum efficiency is expressed as:

[0020]

[0021] Among them, A represents the allocation of resource blocks, p represents the vehicle transmission power, f is the observation frequency, which represents the frequency of sensor nodes observing and sending observation results to the vehicle computing processing center per unit time; α, β, a, b are weight factors, I, and SE are the perception accuracy, average information age, and system spectral efficiency, respectively.

[0022] Furthermore, in step S3, the temporal convolutional network (TCN) is used to predict the resource requirements of network slices in a large time scale to implement resource allocation between slices, specifically including:

[0023] Use TCN to predict the resource requirements of network slices. The input set is in represents the bandwidth resources actually allocated to slice n at the t-1th time scale; represents the resource requirements of slice n predicted at the tth time scale.

[0024] Based on the predicted resource demand, resources are requested from the central controller, which then allocates resources to slices. The predicted slice resources provide a benchmark for the next stage, namely the resource scheduling stage.

[0025] For long-term resource allocation problems, the goal is to minimize the predicted value With actual value The mean square error MSE between:

[0026] P1:

[0027] Among them, m represents the mth large time scale, M is the total number of large time scales, and the bandwidth resources allocated to slice n are obtained Then calculate the number of resource blocks in slice n, B represents the bandwidth of a resource block.

[0028] Furthermore, in step S3, deep deterministic policy gradient (DDPG) is used to schedule resource blocks to vehicle users in a small time scale to determine a resource block allocation method that satisfies various constraints and maximizes digital twin fidelity and system spectrum efficiency. Specifically, the following steps are included:

[0029] Step 1: Initialize the evaluation network, target network, experience replay pool, number of samples, batch size, and update frequency;

[0030] Step 2: Use TCN to predict the resource requirements of each slice n1, n2, and N Allocate bandwidth resources to each slice as the current environment state;

[0031] Step 3: Select action a based on the state t ={A1(t),A2(t),...,A I (t),p1(t),p2(t),...,p I (t)}, where A1(t), A2(t), ..., A I (t) represents the allocation of resource blocks for vehicle I at time t; p1(t), p2(t), ..., p I (t) represents the power distribution of vehicle I at time t; execute action a t And get reward t And observe the new state s t+1 , and convert the quaternion (s t ,a t ,r t ,s t+1 ) into the experience replay pool;

[0032] Step 4: Randomly sample B (s) from the experience replay pool i ,a i ,r i ,s i+1 ), calculate the target value and update the Critic network;

[0033] Step 5: Update the Target network through soft update;

[0034] Step 6: Repeat steps 3 to 5 until the rewards converge, indicating that the optimal resource block resource and power allocation scheme has been found.

[0035] The beneficial effects of the present invention are: the present invention constructs a vehicle network slicing architecture based on digital twins to realize real-time monitoring of vehicles and network resources. In order to ensure the fidelity of the digital twin network, it is divided into three parts: the fidelity of the perception process, the fidelity of the data transmission process, and the fidelity of the information age assurance; under the condition of ensuring vehicle QoS, an optimization model is established with the goal of maximizing the fidelity of digital twins and system spectrum efficiency. It is decomposed into two sub-problems on the time scale, and a dual-time-scale network slice resource allocation strategy is proposed, namely, a TCN-DDPG resource allocation method; in the large time scale, TCN is used to predict the resource requirements of network slices to realize resource allocation between slices; in the small time scale, deep DDPG is used to schedule resource blocks to vehicle users.

[0036] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0038] Figure 1 This is a diagram of the Internet of Vehicles slicing architecture based on digital twins;

[0039] Figure 2 It is a diagram of the data transmission mechanism;

[0040] Figure 3 This is a diagram showing the changing process of information age;

[0041] Figure 4 This is the TCN causal convolution and dilated convolution network diagram;

[0042] Figure 5 This is the DDPG model structure diagram for Internet of Vehicles network slicing. DETAILED DESCRIPTION

[0043] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0044] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0045] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0046] See also Figures 1 to 5 , an embodiment of the present invention provides a method for allocating network slice resources on a dual-time scale in an Internet of Vehicles (IoV) based on digital twins, comprising the following steps:

[0047] (1) Build a network slicing architecture for the Internet of Vehicles based on digital twins, and achieve real-time monitoring of vehicles, base stations, and network resources by building digital twins of physical entities in digital twin servers;

[0048] In specific implementation, the physical layer includes vehicles, base stations, and central controllers; the digital twin layer includes data warehouses, basic models, and functional models; and the application layer includes various vehicle applications, including:

[0049] The data warehouse is used to collect real-time data of physical entities, including vehicle location, speed and power, service requests, base station location and bandwidth, number of slices and allocated RB resources;

[0050] The basic model uses data from the data warehouse to establish digital twins, including vehicle twins, base station twins, and slice twins, thereby constructing a digital twin network for Internet of Vehicles network slices;

[0051] The functional model is used to form a closed loop with the basic model, and generates resource allocation actions according to the current state and sends them to the basic model for policy verification.

[0052] (2) In order to ensure the fidelity of the digital twin network, it is divided into three parts: the fidelity of the perception process, the fidelity of the data transmission process, and the fidelity of the information age guarantee, so as to establish a high-fidelity digital twin network; the fidelity of the perception process is measured by the perception accuracy of the vehicle sensor. The perception data is estimated through the Kalman filter. The filtering error is the difference between the filtered state and the true state. It is assumed that the data observed by the vehicle sensor is sent to the on-board computing processing center and the state is estimated to obtain the true state S at time t. t and filter status Then the state estimation error is The observation frequency f is the frequency at which the sensor node observes and sends the observation results to the vehicle-mounted computing processing center in unit time. In time T, the node sends T·f times of data. Assuming that the signal source is Gaussian, the expectation E[S] = 0, and the variance At the same time, S1, S Tf They are correlated in the time domain and can be described by joint Gaussian random variables, with E[S t ]=0, For any two times t1 and t2, and The covariance of in, At time t1 and t2 and Correlation coefficient, 0<ρ (i,j) <1, can be determined by the characteristics of the measured event, using a general energy index correlation model: Among them, ij is the difference between two observation moments, θ is the energy index related model parameter, and the perception accuracy I(f) is expressed as: Among them, δ is determined by the information source, process noise and observation noise. Since the sensor observes data, it consumes a certain amount of energy. In this paper, it is assumed that the energy consumed by the sensor to observe 1 bit of data is E1. The sensing node observes d bits each time. Then, within time T, the energy consumption of the sensing node can be expressed as The minimum value of the observation frequency f * Can be obtained by f * =arg{min[I(f)>I min ]} to determine, where I min is the threshold of perception accuracy. The relationship model between perception accuracy I and perception frequency f and the relationship model between energy consumption E and perception frequency f proposed in this invention can achieve the goal of obtaining the required perception accuracy with minimum energy consumption. Figure 2 As shown, the vehicle sensor perceives the data and performs state estimation to obtain the optimal estimated state data. Since the difference between the perception data at the current moment and the previous moment may be very small, in order to reduce the pressure of network communication, the data sent to the DT server is the state change After receiving the perception data, the DT server calculates and updates the digital twin, f ji It represents the computing resources allocated by the digital twin server to the vehicle terminal. After the twin is updated, the server will return a feedback signal ACK. After receiving the feedback signal, the vehicle terminal will transmit the next perception data. If there is an error in the received data packet, the data packet will be discarded and the error data packet will be fed back to the terminal to resend the error data packet. The wireless communication rate from the vehicle to the base station can be expressed as Among them, r B is the transmission rate that can be achieved on one RB, Represents the allocation of RBs. The kth RB is allocated to the i-th vehicle at time slot t. otherwise but The transmission latency from vehicle i to the base station can be expressed as The error in data transmission can be measured by the signal-to-noise ratio. i (t) is less than the threshold Γ th , the received information cannot be successfully decoded, that is like Figure 3 As shown, the age of information at time t is defined as the age of information from the time stamp t i The time of the most recently received packet i is given by Δt=t-max{t i :t' i ≤t}, the average information age in the time interval τ can be equivalent to finding the area of ​​the graph, Q i Expressed as the area of ​​an isosceles trapezoid, Y i is the arrival time interval, T i is the delay from sending data to receiving data, then Q i It can be expressed by delay and arrival interval:

[0053] When the time interval τ tends to infinity, the average information age can be expressed as:

[0054]

[0055] Where λ is the packet arrival rate.

[0056] (3) Under the constraints of bandwidth, frequency, power signal-to-interference-and-noise ratio, QoS requirements, and RB allocation, a maximization model is constructed by comprehensively considering the fidelity of the digital twin and the system spectrum efficiency. Although allocating more resource blocks will make it easier to meet the QoS requirements of the terminal and increase the system spectrum efficiency, it will also increase the communication pressure in the network and easily cause data accumulation in the queue, thereby increasing the system information age. Therefore, a compromise between the two needs to be considered. The maximization model can be expressed as:

[0057]

[0058] Among them, α, β, a, b are weight factors, I, and SE are the perceptual accuracy, mean information age, and spectral efficiency, respectively.

[0059] The optimization objective is subject to constraints C1-C8 to ensure the effectiveness of the optimization objective. Constraint C1 ensures that the bandwidth allocated to the slice does not exceed the total bandwidth W; constraint C2 ensures that the observation frequency does not exceed the maximum observation frequency f max ; Constraint C3 ensures that the vehicle transmission power does not exceed the maximum value p max ; Constraint C4 ensures that the signal-to-interference-and-noise ratio is less than the threshold; Constraint C5 ensures that the resource block bandwidth allocated to the vehicle by the slice does not exceed the bandwidth allocated to the slice; Constraint C6 ensures that any two slices cannot use the same RB at the same time; Constraint C7 ensures the QoS requirements of the vehicle; Constraint C8 is the RB allocation constraint.

[0060] C1:

[0061] C2:0≤f≤f max

[0062] C3:0≤p≤p max

[0063] C4:SINR i (t)<Γ th

[0064] C5:

[0065] C6:

[0066] C7:

[0067] C8:

[0068] To address the long-term fluctuations in vehicle service requests, TCN is used to effectively predict user mobility and leverage patterns in historical data to perform long-term resource allocation. To address the vulnerability of vehicular networks to environmental changes, finer-grained resource allocation is implemented, using the DDPG algorithm, which can handle continuous motion problems, to track vehicle mobility.

[0069] (4) Figure 4 The following shows the TCN causal convolution and dilated convolution network. TCN is used to predict the resource requirements of network slices. The input set is Predicting resource requirements for slices Where, represents the bandwidth resources actually allocated to slice n at the t-1th time scale; Represents the resource demand of slice n predicted at the tth time scale. Based on the predicted resource demand, the central controller is requested for resources, and then the controller allocates resources to slices. The predicted slice resources provide a benchmark for the next stage, the resource scheduling stage. For the long-term resource allocation problem, the goal is to minimize the predicted value With actual value The mean square error MSE between:

[0070] P1:

[0071] Where m represents the mth large time scale and M is the total number of large time scales. Get the bandwidth resources allocated to slice n Then the number of RBs in slice n can be calculated. B represents the bandwidth of one RB.

[0072] (5) After allocating dedicated resources to each slice, an online resource scheduling algorithm is required to calibrate the allocated resources so that the slices can adapt to the dynamic environment. For the network slice resource allocation model with the objective function J, it can be converted into a model-free Markov decision process. The state space and action space of this MDP are both of high dimensions and can be represented by a four-tuple (S, A, P, R), where S represents the state space consisting of all possible states of the agent, A represents the set of all possible actions that the agent can take, P represents the transition probability, and R represents the reward function. State: For the resource scheduling problem, due to the changes in network load caused by random requests, the real-time link situation is not suitable for the environment state. If the real-time link is chosen as the state, the uncertainty of the number of links will lead to a dimension mismatch in the input. Therefore, the state is defined as: Action: Scheduling resources for each vehicle and controlling the vehicle's transmission power, defined as: a t ={A1(t),A2(t),...,A I (t),p1(t),p2(t),...,p I (t)}, reward function: R represents the reward returned after the agent interacts with the environment. Generally speaking, this reward return can directly reflect the quality of the selected action. Considering the constraints, the reward function is defined as like Figure 5 The following is the DDPG structure of the Internet of Vehicles network slice. This step specifically includes the following steps:

[0073] Step 1: Initialize the evaluation network, target network, experience replay pool, number of samples, batch size, update frequency, etc.

[0074] Step 2: Use TCN to predict the resource requirements of each slice n1, n2, ..., N Allocate bandwidth resources to each slice as the current environment state.

[0075] Step 3: Select action a based on the status t ={A1(t),A2(t),...,A I (t),p1(t),p2(t),...,p I (t)}, perform action a t And get reward t And observe the new state s t+1 , and convert the quaternion (s t ,a t ,r t ,s t+1 ) into the experience replay pool.

[0076] Step 4: Randomly sample B (s) from the experience replay pool i ,a i ,r i ,s i+1 ), calculate the target value, and update the Critic network.

[0077] Step 5: Update the Target network using soft update.

[0078] Step 6: Repeat steps 3 to 5 until the rewards converge, indicating that the optimal resource block (RB) and power resource allocation scheme has been found.

[0079] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A dual-time-scale network slice resource allocation method for Internet of Vehicles based on digital twins, characterized by: The following steps are involved: S1: Build a digital twin-based IoV network slicing architecture. By building digital twins of physical entities in digital twin servers, real-time monitoring of vehicles, base stations, and network resources is achieved. S2: Divide the digital twin network into three parts: the fidelity of the perception process, the fidelity of the data transmission process, and the fidelity of the information age guarantee, in order to establish a high-fidelity digital twin network; S3: Establish an optimization model that maximizes digital twin fidelity and system spectrum efficiency while ensuring vehicle quality of service (QoS). This model is decomposed into two sub-problems on a time scale, and a deep deterministic policy gradient (TCN-DDPG) resource allocation method based on a temporal convolutional network (TCN) is proposed. On a large time scale, a temporal convolutional network (TCN) is used to predict the resource requirements of network slices and implement resource allocation between slices. On a small time scale, a deep deterministic policy gradient (DDPG) is used to schedule resource blocks to vehicle users. This method is used to determine resource block allocation that satisfies various constraints and maximizes digital twin fidelity and system spectrum efficiency. The optimization model described in step S3, which aims to maximize the fidelity of the digital twin and the system spectrum efficiency, is expressed as: in, Indicates the allocation of resource blocks, Represents the vehicle's transmission power, The observation frequency indicates the frequency of sensor nodes observing and sending observation results to the vehicle-mounted computing and processing center per unit time; 、 、 、 is the weight factor, 、 and They are perception accuracy, average information age and system spectral efficiency respectively; Constraints include: The constraint C1 ensures that the bandwidth allocated to the slice does not exceed the total bandwidth , Indicates in m A large time scale is assigned to the slice bandwidth resources, M is the total number of large time scales; the constraint C2 ensures the observation frequency f Not exceeding the maximum observation frequency ; Constraint C3 ensures that the vehicle's transmission power does not exceed the maximum value Constraint C4 ensures that the signal-to-interference-noise ratio is less than the threshold; Constraint C5 ensures that the bandwidth of the resource block allocated to the vehicle does not exceed the bandwidth allocated to the slice. Indicates the allocation of resource blocks. When slicing n Middle Resource blocks are allocated to the first Vehicle, then =1, otherwise =0; Constraint C6 ensures that any two slices cannot use the same resource block at the same time; Constraint C7 ensures the QoS requirements of the vehicle; Constraint C8 is the allocation constraint of the resource block; In the large time scale, the temporal convolutional network (TCN) is used to predict the resource requirements of network slices and implement resource allocation between slices. Specifically, the following steps are involved: Use TCN to predict the resource requirements of network slices. The input set is ,in Indicates the The long time scale actually assigned to the slice bandwidth resources; Representatives in the Slices of long timescale forecasts resource needs; Based on the predicted resource demand, resources are requested from the central controller, which then allocates resources to slices. The predicted slice resources provide a benchmark for the next stage, namely the resource scheduling stage. For long-term resource allocation problems, the goal is to minimize the predicted value With actual value The mean square error MSE between: in, Indicates the A large time scale, The total number of large time scales is assigned to the slice Bandwidth resources Then calculate the slice The number of resource blocks in , B represents the bandwidth of a resource block; The method uses deep deterministic policy gradient (DDPG) to schedule resource blocks to vehicle users in a small time scale. The method is used to decide on a resource block allocation method that satisfies various constraints and maximizes digital twin fidelity and system spectrum efficiency. The method specifically includes the following steps: Step 1: Initialize the evaluation network, target network, experience replay pool, number of samples, batch size, and update frequency; Step 2: Predict each slice through TCN Resource requirements ,allocate bandwidth resources to each slice as the current environment state; Step 3: Select an action based on the status ,in express A car in Allocation of time resource blocks; express A car in Power distribution at each moment; execution of actions and get rewards And observe the new state , and the quadruple Put it into the experience replay pool; Step 4: Randomly sample from the experience replay pool indivual , calculate the target value and update the Critic network; Step 5: Update the Target network through soft update; Step 6: Repeat steps 3 to 5 until the rewards converge, indicating that the optimal resource block resource and power allocation scheme has been found.

2. The method for allocating dual-time-scale network slice resources for Internet of Vehicles based on digital twins according to claim 1 is characterized by: In step S1: the Internet of Vehicles network slicing architecture based on digital twins includes a physical layer, a digital twin layer, and an application layer; the physical layer includes vehicles, base stations, and a central controller; the digital twin layer includes a data warehouse, a basic model, and a functional model; the application layer includes various vehicle applications, wherein: The data warehouse is used to collect and store real-time data of physical entities; The basic model establishes a digital twin from the data stored in the data warehouse, thereby constructing a digital twin network of the Internet of Vehicles network slice; The functional model is used to form a closed loop with the basic model to perform strategy verification.

3. The method for allocating dual-time-scale network slice resources for Internet of Vehicles based on digital twins according to claim 1 is characterized by: The step S2 specifically includes: The fidelity of the perception process is measured by perception accuracy. The greater the perception frequency, the better the perception accuracy, while the perception energy consumption increases. The optimal perception accuracy and the minimum perception energy consumption solution are obtained by combining the perception accuracy and the perception energy consumption. The fidelity of the data transmission process is measured by the signal-to-noise ratio during the transmission process. If the signal-to-noise ratio is less than a threshold, the information will be erroneous. The fidelity of the information age guarantee is measured by the information age of the transmitted information. Maximizing the fidelity of the information age guarantee means minimizing the information age of the transmitted information.

Images