Committee computing resource allocation and vehicle digital twinborn deployment method in Internet of Vehicles

Abstract

The invention relates to a communication computing resource allocation and vehicle digital twinning deployment method in the Internet of Vehicles, and belongs to the technical field of mobile communication. The method comprises the following steps: S1, constructing a digital twin-assisted Internet of Vehicles communication, inductance and calculation integrated network; s2, designing a vehicle dynamic frame based on the sensing integration technology; s3, constructing a utility evaluation function of the vehicle perception data, and solving perception information utility; s4, constructing a vehicle digital twinborn deployment strategy, and calculating vehicle digital twinborn deployment cost and migration effectiveness; and S5, establishing an optimization model for minimizing the perceptual data information age and the vehicle digital twinborn deployment and migration cost, and solving an optimal vehicle sensing calculation resource allocation and vehicle digital twinborn deployment scheme by adopting multi-agent deep reinforcement learning. Through the flexible dynamic frame design and the vehicle digital twinborn deployment strategy, the average information age of the sensing data and the digital twinborn migration rate are reduced, and the sensing data effectiveness and the edge computing resource load balance are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile communications and relates to a method for allocating telematics resources and deploying vehicle digital twins in an Internet of Vehicles. Background Art

[0002] Integrated Sensing, Communication and Computing (ISCC) is one of the important technologies for realizing intelligent transportation. The main implementation scheme is to combine ISCC with mobile edge computing. ISCC realizes the dual functions of perception and communication by sharing wireless spectrum resources. Mobile edge computing (MEC) can provide computing and storage resources for mobile vehicles and reduce computing latency by sinking computing power to the edge.

[0003] Digital Twin (DT) technology is a technology that maps physical entities to digital space in real time. It can capture the dynamic state information of physical entities in real time and provide computing and communication resources in physical space quickly and accurately according to user requirements. Deploying digital twins on edge servers can achieve low-latency interaction with their corresponding entities.

[0004] When using synaesthesia integrated equipment, although a long perception time can ensure accurate and effective perception, a large amount of perception data will produce a longer transmission delay and reduce the data utility, while a perception time that is too short may lead to inaccurate perception and fail to ensure safety. How to balance the perception and communication time resource allocation in synaesthesia integration remains to be solved. Similarly, edge computing servers usually provide services to multiple end users, and multiple vehicles may compete for the same server resources while other servers are idle. How to reasonably allocate server resources to end users to ensure effective load balancing remains to be solved. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a method for allocating communication perception resources and deploying vehicle digital twins in the Internet of Vehicles, so as to achieve optimal allocation of vehicle communication perception resources and optimal deployment of digital twins, thereby maximizing the utility of perception data and load balancing of edge computing resources, and improving the system service quality.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for allocating synaesthesia computing resources and deploying vehicle digital twins in a vehicle networking scenario, comprising:

[0008] S1: Build a digital twin-assisted IoV integrated network;

[0009] S2: Design the vehicle dynamic frame structure based on synaesthesia integration technology;

[0010] S3: Construct the utility evaluation function of vehicle perception data and solve the utility of perception information;

[0011] S4: Construct the vehicle digital twin deployment strategy and calculate the vehicle digital twin deployment cost and migration utility;

[0012] S5: Establish an optimization model to minimize the information age of perception data and the deployment and migration cost of vehicle digital twins;

[0013] S6: Use multi-agent deep reinforcement learning to solve the optimal synaesthesia resource allocation and vehicle digital twin deployment solution.

[0014] Furthermore, in S1, the proposed digital twin-assisted vehicle network integrated sensing and computing network includes two layers: the terminal layer and the edge layer. The terminal layer in S1 consists of vehicles with limited computing and storage capabilities, and the set of vehicle n is recorded as N = {1,2,3,...,n}. The edge layer consists of a base that provides access services and an edge server that provides computing and storage services. Each base station is equipped with an edge server. The set of base stations v is recorded as V = {1,2,...,v}, and the set of edge servers m is recorded as M = {1,2,3,...,m}. Consider a long-lifetime time slot set K, the twin information of each vehicle is updated in its single time slot, and the set of time slot k is set to K = {1,2,..,k}. In each time slot, the vehicle and base station update their status.

[0015] Furthermore, in S2, following the current mainstream usage scheme for synaesthesia integrated devices, a time-division dynamic frame structure based on 5GNR (5th Generation New Radio, 5GNR) is used. Compared with 4GLTE (4th Generation Long-Term Evolution, 4GLTE), the frame time slot can be adjusted, allowing the vehicle to dynamically adjust the communication and perception duration ratio according to demand.

[0016] Furthermore, in S2, the vehicle uses orthogonal frequency division multiplexing (OFDM) signals for perception. The conditional mutual information between the impulse response and the echo signal is used to evaluate the perception accuracy. The throughput is used to measure the communication performance between the vehicle and the base station.

[0017] Furthermore, in S3, the data utility of the perception data at different locations is calculated according to the age of information (AoI). The vehicle perception information transmission process includes three parts: the vehicle perceives, uploads the perception information to the base station, and the base station forwards the information to the edge server where the digital twin is deployed. A nonlinear function is used to simulate the utility of perception information for the age of information. The specific utility function is defined as u(t) = 1 / t 2 According to the evolution of information age at each location, the average information age utility of the perception data is obtained by integration, and the utility Q based on information age of vehicle n is obtained by accumulation. u .

[0018] Further, in S4, the matrix X=[x nm ] represents the association relationship between vehicle n and edge server m; if x nm =1, then the digital twin (DigitalTwin, DT) DT of vehicle n is deployed at edge server m. The association matrix is ​​expressed as: x represents the association between vehicle n and edge server m, and n belongs to a vehicle nm in the vehicle set N.

[0019] Vehicle, m belongs to one of the edge servers in the edge server set M.

[0020] Furthermore, in S4, the DT deployment strategy will produce different delays and energy consumption. Specifically, it includes the wired transmission delay of the perception data between the base station and the edge server, the wired transmission delay of the user's DT data uploaded to the edge server through the base station, and the delay caused by the ES processing the user's uploaded data. Let F be the delay required to transmit one unit of data in each unit distance. Then the wired transmission delay from the nearby base station v of the perception data vehicle n to the edge server m where its digital twin is deployed is:

[0021]

[0022] in, It is the amount of perception data generated during the perception process.

[0023] set up is the constructed DT data that vehicle n needs to upload in time slot k. Under the above deployment strategy, the synchronization delay of DT transmitted from the vehicle to the edge server m through the base station v is

[0024]

[0025] p v,m represents the transmission power from base station v to edge server m. Without considering the loss, the transmission energy consumption is expressed as:

[0026]

[0027] The number of user DTs served by each edge server cannot exceed its storage and computing load. For edge server m, its storage size is C m , the computing resources are F m , the storage size occupied by vehicle n deployment DT is set to The computational resources required to process the vehicle task are set to The delay required to process the uploaded data is:

[0028]

[0029] Furthermore, in S4, the deployment cost of the vehicle digital twin is:

[0030]

[0031] Among them, T s Indicates the length of a time slot.

[0032] Furthermore, in S4, the migration strategy is represented by x' n , specifically expressed as x' n =1, vehicle n triggers DT migration, otherwise it does not migrate. The entire migration strategy matrix is ​​expressed as: x' = (x'1, x'2, ..., x' n )

[0033] The vehicle migration utility function U(k) is defined to quantify the impact of vehicle DT migration at time slot k based on delay and energy consumption; the server node energy consumption when the DT of vehicle n migrates from edge server m to edge server j is expressed as:

[0034]

[0035] Among them, g j (k) is a binary variable indicating whether edge server j is active. If it is active, it is represented by g j (k) = 1. Otherwise, it is expressed as g j (k) = 0; represents the computing resource utilization of edge server m; F m is the computing resource of edge server m; represents the state transition energy consumption of edge server j at time slot k.

[0036] The vehicle utility function is expressed as:

[0037]

[0038] in, represents the impact of vehicle delay satisfaction on utility, μ n represents the delay requirement of vehicle n, T n (k) represents the actual interaction delay of vehicle n, μ n -T n (k) increases, It also increases, indicating that the utility of vehicle n increases. γ∈(0,1) is the weight factor, represents the digital twin migration utility of vehicle n.

[0039] Furthermore, in S5, the proposed optimization model for minimizing the age of perception data information and the deployment and migration cost of vehicle digital twins is expressed as:

[0040]

[0041]

[0042]

[0043]

[0044]

[0045]

[0046]

[0047]

[0048]

[0049]

[0050]

[0051] C11:ψ1+ψ2+ψ3=1,ψ1,ψ2,ψ3∈(0,1)

[0052] Q u , Q pl , Q trans They represent the perceived data utility, digital twin deployment cost, and digital twin migration utility respectively; τ n With α n They represent the duration of perception and communication within a single time slot of the vehicle respectively; represents the vehicle perception mutual information, Indicates the communication uplink rate between the vehicle and the base station; represents the vehicle perception frequency; τ n ,α n ,βn They represent the vehicle perception time, communication transmission time, and wired transmission delay between base station and edge server respectively; T s is the duration of a time slot; Indicates the computing resource utilization of the edge server; They represent the computing resources and storage resources required for the vehicle digital twin respectively; F m ,C m Respectively represent the total amount of computing and storage resources of the edge server; ψ1, ψ2, ψ3 represent weight factors respectively. Base station constraint C1 ensures the vehicle perception quality, constraint C2 indicates that the uplink communication link rate cannot exceed the maximum value when the vehicle communicates with the base station, constraint C3 indicates that the vehicle's perception frequency cannot exceed the maximum perception frequency, constraint C4 indicates that the total system delay cannot exceed the length of one frame, constraint C5 indicates that the computing resource utilization of the edge server is less than 1, constraints C6 and C7 ensure that the vehicle's DT can only be deployed on one edge server, constraints C8 and C9 respectively indicate that the vehicle's DT deployment and computing requirements cannot exceed the server's maximum cache and computing space. Constraint C10 indicates that the vehicle's storage and computing resource requirements for the edge server are not negative, and constraint C11 represents the connection between the weight factors.

[0053] Furthermore, in S6, the specific steps for solving the optimal vehicle perception communication resource allocation and digital twin deployment method are as follows: first, the objective function is modeled as a multi-agent partially observable Markov decision process, and the base station in the scene is regarded as an agent. The specific elements are defined as follows:

[0054] (1) State: The global state space S includes all vehicle state information, all vehicle DT deployment information, and the resource utilization of all edge servers. The vehicle state information includes the data to be transmitted by the vehicle and the available communication transmission rate between the vehicle and the base station, as well as the AoI during the vehicle transmission process;

[0055] (2) Action space: Each agent selects its actions, including the communication and perception time allocation of vehicles within the agent’s coverage area and the vehicle DT deployment strategy;

[0056] (3) State transition probability: After the agent performs an action on the environment in the current state, the environment state will transfer to the next state with a certain probability;

[0057] (4) Reward function: The reward function should be designed to enable each agent to make optimal decisions on DT deployment and synaesthesia resource allocation, and is therefore designed based on the agent’s contribution to the system.

[0058] Furthermore, in S6, the collaborative learning is completed by adopting the method of "centralized training and distributed execution" of multiple agents to achieve the optimal VDT deployment and communication perception resource allocation strategy. The algorithm training process specifically includes the following steps:

[0059] S61: Initialize Q network parameters and deterministic policy network parameters And experience replay pool D;

[0060] S62: Collect initial status

[0061] S63: Calculate the continuous action parameter τ according to the gradient descent method v (k) and α v (k);

[0062] The formula for the gradient descent method is:

[0063]

[0064] Among them, λ d is the learning rate of the Q network, λ s and λ ve Respectively represent the weights of the local parameters and fusion parameters of the agent;

[0065] It is used to update the network parameters of the target Q network. It is used to update the network parameters of the deterministic policy network;

[0066] S64: Sample an action a with exploration probability ε and probability distribution ψ v,k , with probability 1-ε, select the discrete action value function Maximum discrete motion At the same time, determine the corresponding continuous actions

[0067] τ v (k),α v (k). Where s v,k Indicates the status, represents the deployment strategy of vehicle DT in the edge server,

[0068] τ v (k),α v (k) represent vehicle perception and communication duration respectively.

[0069] S65: Execute action a v,k , get instant reward r v,k and the next state s v,k+1

[0070] S66: Tuple (s v,k,α v,k ,r v,k ,s v,k+1 ) Deposit into the experience replay pool D v

[0071] S67: Calculate the loss function of the Q network and the deterministic policy network based on the loss function and

[0072] The loss functions are:

[0073]

[0074]

[0075] S68: Update Q network parameters according to gradient descent method and

[0076]

[0077]

[0078] where λ s and λ ve are the weights of the local parameters and fusion parameters of the agent respectively;

[0079] S69: Update fusion network parameters: and

[0080]

[0081] Among them, ← is the update symbol, which means updating the calculated fusion network parameters. is the updated parameter of the Q network at the next moment, Update the parameters of the deterministic policy network for the next time step. represents the update gradient of the deterministic network parameters, is the update gradient of the Q network parameters, λ v ' e is the updated fusion network parameter weight. represents the global loss function of the fusion network, represents the expectation about the strategy, y ve represents the target value of the fusion network, Q me represents the global Q function.

[0082] S610: Network parameters are updated and sent to each agent.

[0083] The beneficial effects of the present invention are:

[0084] (1) The dynamic frame structure designed based on the synaesthesia integration technology of the present invention can dynamically adjust the proportion of vehicle perception and communication time. The proposed perception data utility function based on information age can improve the perception data utility while ensuring the communication quality.

[0085] (2) The vehicle digital twin deployment strategy based on data transmission energy consumption and delay evaluation proposed in the present invention can reasonably deploy the vehicle digital twin in the edge server, reasonably allocate edge computing resources, and avoid edge load congestion.

[0086] (3) The digital twin migration strategy proposed in the present invention, which is based on the user digital twin migration utility function evaluation constructed based on migration delay and energy consumption, effectively reduces the digital twin migration cost.

[0087] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0089] Figure 1 The network architecture diagram of the integrated sensing and computing of the Internet of Vehicles assisted by digital twins;

[0090] Figure 2 It is a dynamic frame structure designed based on synaesthesia integration;

[0091] Figure 3 A training flowchart for the communication perception computing resources and vehicle digital twin deployment method for the Internet of Vehicles scenario;

[0092] Figure 4 A block diagram of multi-agent deep reinforcement learning based on centralized training and distributed execution. DETAILED DESCRIPTION

[0093] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0094] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0095] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0096] The network architecture diagram of the vehicle network integrated with sensing and computing assisted by a digital twin described in this embodiment is as follows: Figure 1 As shown, the communication perception computing resource allocation and vehicle digital twin deployment method for the Internet of Vehicles scenario is as follows Figure 2 As shown, the implementation of this method specifically includes the following steps:

[0097] S1: Build a digital twin-assisted IoV integrated network;

[0098] S2: Design the vehicle dynamic frame structure based on synaesthesia integration technology;

[0099] S3: Construct the information age utility function of vehicle perception data and solve the perception information utility;

[0100] S4: Construct the vehicle digital twin deployment strategy and calculate the vehicle digital twin deployment cost twin migration utility;

[0101] S5: Establish an optimization model to minimize the information age of perception data and the deployment and migration cost of vehicle digital twins;

[0102] S6: Use multi-agent deep reinforcement learning to solve the optimal synaesthesia resource allocation and vehicle digital twin deployment scheme. In S1 above, the proposed digital twin-assisted vehicle network synaesthesia integrated network includes two layers: terminal layer and edge layer. The terminal layer in S1 consists of vehicles with limited computing and storage capabilities, and the set of vehicles n is recorded as N = {1,2,3,...,n}. Each vehicle is equipped with synaesthesia integrated equipment, which has both perception and communication functions, can perceive the surrounding environment, and communicate with the base station wirelessly and upload data. The edge layer in S1 consists of a base that provides access services and an edge server that provides computing and storage services. Each base station is equipped with an edge server. The set of base stations v is recorded as V = {1,2,...,v}, and the set of edge servers m is recorded as M = {1,2,3,...,m}. Consider a long-lifetime time slot set K, the twin information of each vehicle is updated in its single time slot, and the set of time slots k is set to K = {1,2,..,k}. In each time slot, the vehicle and base station update their status.

[0103] In the above S2, a time-division dynamic frame structure based on 5GNR is used. The dynamic frame structure is designed as follows: Figure 3 Compared with 4G LTE, the frame time slot can be adjusted, allowing vehicles to dynamically adjust the communication and perception time ratio according to business needs.

[0104] S21: The vehicle uses orthogonal frequency division multiplexing (OFDM) signals for perception, expressed as

[0105]

[0106] where f c is the center frequency, T s Equal to the time length of the complete OFDM signal; represents the waveform amplitude of device n at time slot k; Represents the phase code of the modulation symbol of device n in time slot k; rect[x] is a matrix function whose value is 1 when 0≤x≤1, otherwise it is 0.

[0107] S22: The conditional mutual information between the impulse response and the echo signal is used to evaluate the perceptual accuracy. MI is written as:

[0108]

[0109] τ sen Represents the perception process, It represents the interference (SINR) generated when vehicle n senses in time slot k. It is specifically expressed as:

[0110]

[0111] S23: Throughput is used to measure the communication performance between the vehicle and the base station. The throughput between vehicle n and base station v at time slot k can be expressed as:

[0112]

[0113] It represents the communication SINR between vehicle n and base station v at time k. It is specifically expressed as:

[0114]

[0115] represents the channel gain between vehicle n and base station v.

[0116] In the above S3, the data utility of the perception data at different locations is calculated according to the information age. The vehicle perception information transmission process includes three parts: the vehicle perceives, uploads the perception information to the base station, and the base station forwards the information to the edge server where the digital twin is deployed.

[0117] S31: Define s(t) to represent the generation time of the most recently received information, until instantaneous time t, specifically expressed as

[0118]

[0119] S32: The AoI at instantaneous time t can be given as

[0120] Δ(t)=ts(t)

[0121] S33: Use a nonlinear function to simulate the utility of perceived information for information age. The specific utility function is defined as u(t) = 1 / t 2 According to the evolution of information age at each position, the average information age utility of the perceived data is obtained by integrating:

[0122] S34: For vehicles:

[0123]

[0124] S35: For the base station server:

[0125]

[0126] S36: For edge servers:

[0127]

[0128] S37: derive the utility function of vehicle n based on information age according to the utility function:

[0129]

[0130] In the above S4, the matrix X=[x nm ] represents the association relationship between vehicle n and edge server m; if x nm =1, then the digital twin (DigitalTwin, DT) DT of vehicle n is deployed at edge server m. The association matrix is ​​expressed as: x nm Represents the association between vehicle n and edge server m, where n belongs to one of the vehicles in the vehicle set N, and m belongs to one of the edge servers in the edge server set M. Deployment strategies will produce different delays and energy consumption, including the wired transmission delay of the perception data between the base station and the edge server, the wired transmission delay of the data constructed by the user from the base station to the edge server, and the delay caused by the ES processing the user uploaded data.

[0131] S42: Let F be the delay required to transmit one unit of data in each unit distance. Then the wired transmission delay from the nearby base station v of the sensing data vehicle n to the edge server m where its digital twin is deployed is:

[0132]

[0133] in, It is the amount of perception data generated during the perception process.

[0134] S43: Set is the constructed DT data that vehicle n needs to upload in time slot k. Under the above deployment strategy, the synchronization delay of DT transmitted from the vehicle to the edge server m through the base station v is

[0135]

[0136] p v,m represents the transmission power from base station v to edge server m. Without considering the loss, the transmission energy consumption is expressed as:

[0137]

[0138] S44: The number of user DTs served by each edge server cannot exceed its storage and computing load. For edge server m, its storage size is C m , the computing resources are F m , the storage size occupied by vehicle n deployment DT is set to The computational resources required to process the vehicle task are set to The delay required to process the uploaded data is:

[0139]

[0140] S45: The deployment cost of the vehicle digital twin is:

[0141]

[0142] Among them, T s Indicates the length of a time slot.

[0143] S46: Migration strategy is represented by x' n , specifically expressed as x' n =1, vehicle n triggers DT migration, otherwise it does not migrate. The entire migration strategy matrix is ​​expressed as:

[0144] x'=(x'1,x'2,…,x' n )

[0145] S47: Define the user migration utility function U(k) to quantify the DT migration impact of the vehicle at time slot k based on latency and energy consumption. The computing resource utilization of edge server m at time slot k can be expressed as

[0146]

[0147] Due to the migration of DTs, the running states of some edge servers may change. For example, suppose edge server j did not run any DT before, but since DTs are migrated to this node, the node will transition from dormant state to active state, resulting in state transition energy consumption. represents the state transition energy consumption of edge server j at time slot k:

[0148]

[0149] G j (k) indicates whether there is a state transition in edge server j, as follows:

[0150]

[0151] Therefore, the server node energy consumption when the DT of vehicle n migrates between edge servers m, j is expressed as:

[0152]

[0153] S48: The utility function can be expressed as:

[0154]

[0155] in, represents the impact of vehicle delay satisfaction on utility, μ n represents the delay requirement of vehicle n, T n (k) represents the actual interaction delay of vehicle n, μ n -T n (k) increases, It also increases, indicating that the utility of vehicle n increases. γ∈(0,1) is the weight factor, represents the digital twin migration utility of vehicle n.

[0156] Further, S5, the proposed objective function for maximizing the utility of perception data and minimizing the deployment and migration cost of vehicle digital twins is expressed as:

[0157]

[0158]

[0159]

[0160]

[0161]

[0162]

[0163]

[0164]

[0165]

[0166]

[0167]

[0168] C11:ψ1+ψ2+ψ3=1,ψ1,ψ2,ψ3∈(0,1)

[0169] Q u , Q pl , Q trans They represent the perceived data utility, digital twin deployment cost, and digital twin migration utility respectively; τ n With α n They represent the duration of perception and communication within a single time slot of the vehicle respectively; represents the vehicle perception mutual information, Indicates the communication uplink rate between the vehicle and the base station; represents the vehicle perception frequency; τ n ,α n,β n They represent the vehicle perception time, communication transmission time, and wired transmission delay between base station and edge server respectively; T s is the duration of a time slot; Indicates the computing resource utilization of the edge server; They represent the computing resources and storage resources required for the vehicle digital twin respectively; F m ,C m Respectively represent the total amount of computing and storage resources of the edge server; ψ1, ψ2, ψ3 represent weight factors respectively. Base station constraint C1 ensures the vehicle perception quality, constraint C2 indicates that the uplink communication link rate cannot exceed the maximum value when the vehicle communicates with the base station, constraint C3 indicates that the vehicle's perception frequency cannot exceed the maximum perception frequency, constraint C4 indicates that the total system delay cannot exceed the length of one frame, constraint C5 indicates that the computing resource utilization of the edge server is less than 1, constraints C6 and C7 ensure that the vehicle's DT can only be deployed on one edge server, constraints C8 and C9 respectively indicate that the vehicle's DT deployment and computing requirements cannot exceed the server's maximum cache and computing space. Constraint C10 indicates that the vehicle's storage and computing resource requirements for the edge server are not negative, and constraint C11 represents the connection between the weight factors.

[0170] Further, S6, the specific steps for solving the optimal vehicle perception communication resource allocation and digital twin deployment method are as follows: first, the objective function is modeled as a multi-agent partially observable Markov decision process, and the base station in the scene is regarded as an agent. The specific elements are defined as follows:

[0171] (1) State: Global state space Includes all vehicle status information n , all vehicle DT deployment information X, and the resource utilization of all edge servers e m The vehicle status information includes the data to be transmitted by the vehicle and the available communication transmission rate between the vehicle and the base station, as well as the AoI during the vehicle transmission process. The definition is as follows

[0172]

[0173] (2) Action space: Each agent chooses its action It includes the communication and perception time allocation of vehicles within the coverage area of ​​the agent and the vehicle DT deployment strategy. The action space of multiple agents can be written as

[0174]

[0175] (3) State transition probability: The agent is in the current state s v,k When performing action a to the environment v,kAfter that, the environment state will transfer to the next state with a certain probability, expressed as

[0176]

[0177] (4) Reward function: The design of the reward function should enable each agent to make the best decision on DT deployment and synaesthesia resource allocation. Therefore, it is designed according to the contribution of the agent to the system. The reward of agent v in time slot k is defined as

[0178]

[0179] Among them, λ n,v is a binary variable representing the relationship between vehicle n and agent j, Q n represents the contribution of vehicle n to the system. Therefore, at time slot k, the global reward of the system is

[0180]

[0181] Furthermore, in S6, the collaborative learning is completed by adopting the method of "centralized training and distributed execution" of multiple agents to achieve the optimal VDT deployment and communication perception resource allocation strategy. The algorithm training process specifically includes the following steps:

[0182] S61: Initialize Q network parameters and deterministic policy network parameters And experience replay pool D;

[0183] S62: Collect initial status

[0184] S63: Calculate the continuous action parameter τ according to the gradient descent method v (k) and α v (k);

[0185] The formula for the gradient descent method is:

[0186]

[0187] Among them, λ d is the learning rate of the Q network, λ s and λ ve Respectively represent the weights of the local parameters and fusion parameters of the agent;

[0188] It is used to update the network parameters of the target Q network. It is used to update the network parameters of the deterministic policy network;

[0189] S64: Sample an action a with exploration probability ε and probability distribution ψ v,k, with probability 1-ε, select the discrete action value function Maximum discrete motion At the same time, determine the corresponding continuous actions

[0190] τ v (k),α v (k). Where s v,k Indicates the status, represents the deployment strategy of vehicle DT in the edge server,

[0191] τ v (k),α v (k) represent vehicle perception and communication duration respectively.

[0192] S65: Execute action a v,k , get instant reward r v,k and the next state s v,k+1

[0193] S66: Tuple (s v,k ,α v,k ,r v,k ,s v,k+1 ) is stored in the experience replay pool

[0194] S67: Calculate the loss function of the Q network and the deterministic policy network based on the loss function and

[0195] The loss functions are:

[0196]

[0197]

[0198] S68: Update Q network parameters according to gradient descent method and

[0199]

[0200]

[0201] where λ s and λ ve are the weights of the local parameters and fusion parameters of the agent respectively;

[0202] S69: Update fusion network parameters: and

[0203]

[0204] Among them, ← is the update symbol, which means updating the calculated fusion network parameters. is the updated parameter of the Q network at the next moment, Update the parameters of the deterministic policy network for the next time step. represents the update gradient of the deterministic network parameters, is the update gradient of the Q network parameters, λ v ' e is the updated fusion network parameter weight. represents the global loss function of the fusion network, E represents the expectation of the strategy, and y ve represents the target value of the fusion network, Q me represents the global Q function.

[0205] S610: Network parameters are updated and sent to each agent.

[0206] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for allocating synaesthesia computing resources and deploying vehicle digital twins in an Internet of Vehicles, characterized by: The method comprises the following steps: S1: Build a digital twin-assisted IoV integrated network; S2: Design the vehicle dynamic frame structure based on synaesthesia integration technology; S3: Construct the utility evaluation function of vehicle perception data and solve the utility of perception information; S4: Construct the vehicle digital twin deployment strategy and calculate the vehicle digital twin deployment cost and digital twin migration utility; S5: Establish an optimization model to minimize the information age of perception data and the deployment and migration cost of vehicle digital twins; S6: Use multi-agent deep reinforcement learning to solve the synaesthesia resource allocation and vehicle digital twin deployment solutions.

2. The method for allocating telematics resources and deploying vehicle digital twins in an Internet of Vehicles according to claim 1, characterized in that: In S1, the proposed digital twin-assisted IoV integrated network includes two layers: a terminal layer and an edge layer; The terminal layer consists of vehicles with limited computing and storage capabilities. The set of vehicles n is denoted as N = {1, 2, 3, ..., n}; The edge layer consists of a base station that provides access services and an edge server that provides computing and storage services. Each base station is equipped with an edge server; the digital twin of the vehicle is deployed in the edge server; the set of base stations v is denoted as V = {1,2,...,v}, and the set of edge servers m is denoted as M = {1,2,3,...,m}; considering a long-lifetime time slot set K, the twin information of each vehicle is updated within its single time slot k, and the set of time slots is set to K = {1,2,..,k}.

3. The method for allocating telematics resources and deploying vehicle digital twins in an Internet of Vehicles according to claim 1, characterized in that: In S2, a time-division dynamic frame structure based on 5GNR (5th Generation New Radio, 5GNR) is used to enable the vehicle to dynamically adjust the ratio of communication and perception time according to business needs; the vehicle uses orthogonal frequency division multiplexing (Orthogonal Frequency Division Multiplexing, OFDM) signals for perception; conditional mutual information and throughput are used to evaluate the perception accuracy and the communication performance between the vehicle and the base station respectively.

4. The method for allocating synaesthesia computing resources and deploying vehicle digital twins in an Internet of Vehicles according to claim 1, characterized in that: In S3, the data utility of the perception data at different locations is evaluated according to the age of information (AoI); The vehicle perception information transmission process includes three parts: the vehicle perceives, uploads the perception information to the base station, and the base station forwards the information to its corresponding edge server where the digital twin is deployed; A nonlinear function is used to simulate the utility of perceived information for information age. The average information age utility of the perceived data is obtained by integrating and summing the evolution of the information age at each location. The specific utility function is defined as u(t) = 1 / t 2 , where variable t is time; the average information age utility of the perception data is obtained by integrating the information age evolution of each location, and the utility Q based on the information age of vehicle n is obtained by accumulating u .

5. The method for allocating telematics resources and deploying vehicle digital twins in an Internet of Vehicles according to claim 1, characterized in that: In S4, the matrix X=[x nm ] represents the association relationship between vehicle n and edge server m; if x nm =1, then the digital twin (DT) of vehicle n is deployed at edge server m; the association matrix is ​​expressed as: x nm Represents the association relationship between vehicle n and edge server m, where n belongs to one of the vehicles in the vehicle set N, and m belongs to one of the edge servers in the edge server set M; the initial digital twin deployment strategy is to deploy the vehicle digital twin at the edge server closest to it; DT deployment strategies will produce different delays and energy consumption, including the wired transmission delay of the perception data between the base station and the edge server, the wired transmission delay of the data used to build the DT by the vehicle and uploaded to the edge server through the base station, and the delay caused by the ES processing the data uploaded by the vehicle. Let F be the delay required to transmit one unit of data in each unit distance, and d(v,m) represents the wired transmission distance between the base station v and the edge server m. Then the wired transmission delay of the perception data from the base station v near the vehicle n to the edge server m where its digital twin is deployed is: in, The amount of perception data generated during the perception process; set up For the constructed DT data that vehicle n needs to upload in time slot k, the synchronization delay of DT transmitted from the vehicle to the edge server m through the base station v under the above deployment strategy is: p v,m represents the transmission power from base station v to edge server m. Without considering the loss, the transmission energy consumption is expressed as: The number of vehicles DT served by each edge server cannot exceed its storage and computing load. For edge server m, its storage size is C m , the computing resources are F m , the storage size occupied by vehicle n deployment DT is set to The computational resources required to process the vehicle task are set to The delay required to process the uploaded data is: The deployment cost of a vehicle digital twin is: Among them, T s Indicates the length of a time slot; The migration strategy is denoted as x' n , specifically expressed as x' n =1, vehicle n triggers DT migration, otherwise it does not migrate; the entire migration strategy matrix is ​​expressed as: x'=(x'1,x'2,...,x' n ) The vehicle migration utility function U(k) is defined to quantify the impact of vehicle DT migration at time slot k based on delay and energy consumption; the server node energy consumption when the DT of vehicle n migrates from edge server m to edge server j is expressed as: Among them, g j (k) is a binary variable indicating whether edge server j is active. If it is active, it is represented by g j (k) = 1; otherwise, it is expressed as g j (k) = 0; represents the computing resource utilization of edge server m; F m is the computing resource of edge server m; represents the state transition energy consumption of edge server j at time slot k; The vehicle utility function is expressed as: in, represents the impact of vehicle delay satisfaction on utility, μ n represents the delay requirement of vehicle n, T n (k) represents the actual interaction delay of vehicle n, When increasing, It also increases, indicating that the utility of vehicle n increases; γ∈(0,1) is the weight factor, represents the digital twin migration utility of vehicle n.

6. The method for allocating telematics resources and deploying vehicle digital twins in an Internet of Vehicles according to claim 1, characterized in that: In S5, the proposed optimization model for minimizing the age of perception data information and the deployment and migration cost of vehicle digital twins is expressed as: C11:ψ1+ψ2+ψ3=1,ψ1,ψ2,ψ3∈(0,1) They represent the perception data utility of vehicle n, the digital twin deployment cost, and the digital twin migration utility respectively; τ n represents the duration of vehicle perception in a single time slot, α n Indicates the communication duration of a vehicle in a single time slot; represents the vehicle perception mutual information, Indicates the communication uplink rate between the vehicle and the base station; represents the vehicle perception frequency; τ n ,α n ,β n They represent the vehicle perception time, communication transmission time, and wired transmission delay between base station and edge server respectively; T s is the duration of a time slot; Indicates the computing resource utilization of the edge server; represents the computing resources required for the vehicle digital twin, F represents the storage resources required by the vehicle digital twin; m represents the total computing resources of the edge server, C m represents the total storage resources of the edge server; ψ1, ψ2, ψ3 represent weight factors respectively; Base station constraint C1 ensures the vehicle perception quality, constraint C2 indicates that the uplink communication link rate cannot exceed the maximum value when the vehicle communicates with the base station, constraint C3 indicates that the vehicle's perception frequency cannot exceed the maximum perception frequency, constraint C4 indicates that the total system delay cannot exceed the length of one frame, constraint C5 indicates that the computing resource utilization of the edge server is less than 1, constraints C6 and C7 ensure that the vehicle's DT can only be deployed on one edge server, constraints C8 and C9 respectively indicate that the vehicle's DT deployment and computing requirements cannot exceed the server's maximum cache and computing space; constraint C10 indicates that the vehicle's storage and computing resource requirements for the edge server are not negative, and constraint C11 represents the connection between the weight factors.

7. The method for allocating telematics resources and deploying vehicle digital twins in an Internet of Vehicles according to claim 1, characterized in that: In S6, the method for solving the optimal vehicle perception communication resource allocation and digital twin deployment is specifically as follows: first, the objective function is modeled as a multi-agent partially observable Markov decision process, and the base station in the scene is regarded as an agent; the specific elements are defined as follows: (1) State: Global state space Including all vehicle status information, all vehicle DT deployment information, and resource utilization of all edge servers; vehicle status information includes the data to be transmitted by the vehicle and the available communication transmission rate between the vehicle and the base station, as well as the AoI during the vehicle transmission process; (2) Action space: Each agent selects its actions, including the communication and perception time allocation of vehicles within the agent’s coverage area and the vehicle DT deployment strategy; (3) State transition probability: After the agent performs an action on the environment in the current state, the environment state will transfer to the next state with a certain probability; (4) Reward function: The reward function should be designed so that each agent can make the best decision on DT deployment and synaesthesia resource allocation, and therefore should be designed based on the agent’s contribution to the system; The multi-agent "centralized training, distributed execution" approach is used to complete collaborative learning to achieve the optimal VDT deployment and communication perception resource allocation strategy; The algorithm training process specifically includes the following steps: S61: Initialize Q network parameters and deterministic policy network parameters And experience replay pool D; S62: Collect initial status S63: Calculate the continuous action parameter τ according to the gradient descent method v (k) and α v (k); The formula for the gradient descent method is: Among them, λ d is the learning rate of the Q network, λ s and λ ve Respectively represent the weights of the local parameters and fusion parameters of the agent; It is used to update the network parameters of the target Q network. It is used to update the network parameters of the deterministic policy network; S64: Sample an action a with exploration probability ε and probability distribution ψ v,k , with probability 1-ε, select the discrete action value function Maximum discrete motion At the same time, determine the corresponding continuous action τ v (k),α v (k); where s v,k Indicates the status, represents the deployment strategy of vehicle DT in the edge server, τ v (k),α v (k) represents the vehicle perception and communication duration respectively; S65: Execute action a v,k , get instant reward r v,k and the next state s v,k+1 ; S66: Tuple (s v,k ,α v,k ,r v,k ,s v,k+1 ) is stored in the experience replay pool S67: Calculate the loss function of the Q network according to the loss function and the loss function of the deterministic policy network S68: Update Q network parameters according to gradient descent method and where λ s and λ ve are the weights of the local parameters and fusion parameters of the agent respectively; S69: Update fusion network parameters: and Among them, ← is the update symbol, which means updating the calculated fusion network parameters. is the updated parameter of the Q network at the next moment, Update parameters of the deterministic strategy network for the next moment; represents the update gradient of the deterministic network parameters, is the update gradient of the Q network parameters, λ v ' e is the updated fusion network parameter weight; represents the global loss function of the fusion network, represents the expectation about the strategy, y ve represents the target value of the fusion network, Q me represents the global Q function; S610: Network parameters are updated and sent to each agent.

Images