Digital twinning assisted collaborative awareness and edge collaboration resource allocation method
Through the multi-layer collaborative perception framework and edge collaboration method assisted by digital twin, the resource allocation and task offloading of collaborative perception of Internet of Vehicles is optimized, which solves the problems of low resource utilization efficiency and poor system performance in the existing technology, and achieves delay reduction and Improved safety and suitable for autonomous driving applications.
Patent Information
- Application Number
- CN202411091398.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-09
AI Technical Summary
The existing Internet of Vehicles collaborative perception solutions have problems such as low resource utilization efficiency, poor system performance and insufficient security, especially under dynamic network conditions, which leads to the real-time and security of collaborative perception being affected.
Using a multi-layer collaborative perception framework and edge collaboration method with digital twin assist, a global map and simulation model is built through a digital twin system, combining blockchain technology and deep learning algorithm HAS-MADDPG, optimize resource allocation and task offloading to realize the Internet of Vehicles Coordinated perception reduces latency and improves security.
It effectively reduces the execution delay of collaborative perception, improves resource utilization and system performance, and enhances the safety and reliability of autonomous driving.
Smart Images

Figure CN119012392A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of mobile communication technology and relates to a resource allocation method for collaborative perception and edge collaboration assisted by digital twins. Background Art
[0002] The perception capability of a single connected automatic vehicle (CAV) has inherent limitations, and the emergence of collaborative perception technology in the Internet of Vehicles can effectively overcome this problem. By fusing the perception data of multiple CAVs and RSUs, the perception range can be greatly expanded and the perception accuracy can be improved. However, there are still some challenges in the existing single-layer collaborative perception scheme. The first is the inefficient resource utilization. In the early fusion stage, excessive raw data transmission may lead to bandwidth saturation and underutilization of computing power; in the late fusion stage, only sending processed object data may lead to saturation of computing power and underutilization of bandwidth. Even in the mid-term fusion, resource utilization efficiency is often not high under dynamic network conditions. The second is poor system performance. The above-mentioned unbalanced resource utilization will cause the overall system delay to increase, affecting the real-time and safety of collaborative perception. Therefore, it is necessary to design a more flexible and efficient collaborative perception scheme to improve system performance. Therefore, it is urgent to study a collaborative perception framework for the Internet of Vehicles that can dynamically adapt to network conditions and make full use of network resources to meet the needs of applications such as autonomous driving.
[0003] Due to the limited computing power of CAV, CAV often chooses early fusion when sensing data and offloads the data to MES for processing. However, with the explosive growth of data volume, a single MES is difficult to cope with multiple tasks, so a new solution is needed. Edge collaboration makes up for the shortcomings of a single MES in communication, computing and storage resources by distributing processing tasks, saving system power consumption and time overhead. However, with the increase in the number of collaborative nodes, security becomes the most concerned issue for CAV. Some nodes may be attacked or deliberately set as malicious nodes, resulting in information leakage or tampering. Artificial Intelligence (AI) has achieved remarkable results in fields such as intelligent transportation, and many studies have used AI to solve resource allocation and task offloading problems in Mobile Edge Computing (MEC). However, the storage space and computing resource limitations of CAV lead to insufficient training data for AI models, resulting in inaccurate prediction results. In the edge collaboration mode, using DT to train AI models can assist CAV in selecting fusion methods and performing task offloading, thereby improving the overall performance of the network. Therefore, using DT to explore the application of AI in edge node collaboration to help CAVs intelligently offload tasks is an issue worthy of in-depth research, revealing the necessity of exploring new offloading methods. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide a resource allocation method for collaborative perception and edge collaboration assisted by a digital twin, which can optimize the resource allocation strategy of collaborative perception and edge collaboration in the Internet of Vehicles and reduce the execution delay of collaborative perception.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] A resource allocation method for collaborative perception and edge collaboration in a digital twin-assisted Internet of Vehicles includes the following steps:
[0007] S1: In the scenario of connected vehicles, a multi-layer collaborative perception global map framework is proposed to help connected autonomous vehicles (CAVs) obtain the perception results of the global map;
[0008] S2: A digital twin (DT)-assisted edge collaboration approach is proposed to provide offloading services for computationally intensive tasks;
[0009] S3: Analyze the delay of system task execution based on the collaborative perception process;
[0010] S4: A resource allocation scheme with the goal of minimizing the delay in completing collaborative sensing tasks, constructing a joint problem of sensing task allocation, offloading, and computing resource allocation;
[0011] S5: A multi-agent deep deterministic policy gradient HAS-MADDPG algorithm in a hybrid action space is proposed to solve the optimization problem and learn the resource allocation solution with the minimum task completion delay.
[0012] Furthermore, in S1, the IoV scenario consists of a physical IoV system and a digital twin (DT) space. The physical IoV system includes a central macro base station (MBS), several RSUs, MESs, and CAVs. An MES with computing and storage capabilities is deployed near each RSU and is directly connected to the RSU via a wired link. In addition, each CAV and MES is equipped with blockchain technology, which can assist the CAV in selecting a suitable MES for unloading. The CAV is equipped with various sensors and radars to collect data inside and outside the vehicle, such as engine speed and traffic light signals. When the line of sight is obstructed, the CAV can obtain the latest global map data from the digital twin (DT) system to assist driving. MBS plays the role of a digital twin data center (DTDC) in DT-IoV. In addition to providing traditional base station functions, it also stores all DT data within its coverage area. In addition to being able to access the MBS, the vehicle can also unload its own perception data to the MES for processing through a wireless channel. DT space: Each network unit in the physical entity (including CAV, RSU and MES) sends its current operating status to DT through a real-time channel. VE can be constructed in DT space. IoV Digital simulation models for different service levels. DT-IoV can establish a three-level simulation platform. First, DT collects data generated during CAV driving, such as engine speed, fuel / battery remaining, blockchain data, MES communication and computing resources, blockchain data, communication environment information such as channel state information, etc., to build a single entity level simulation model. On this basis, complex hierarchical models between single entities can be further established. For example, in the vehicle-to-infrastructure (V2I) scenario, the DT space not only stores independent data of CAV and RSU, but also creates a dynamic multi-dimensional and multi-time scale virtual model of the communication environment between vehicles and roadside units, which is very similar to the real environment. Based on these models and big data, the DT space can use machine learning algorithms such as MADDPG to provide intelligent solutions for problems such as task offloading and resource allocation in MES scenarios. As more and more physical entities are built in the DT space, a complex system-level DT IoV that is almost consistent with the real IoV world can be formed. The DT IoV can simulate traffic flow control and guidance strategies on the physical IoV system and intuitively evaluate its effects;
[0013] Assume that there are N vehicles and M RSUs in the system. Since a single MES is coupled to one RSU, the number of MESs is also M. RSUs, MESs, CAVs, and the communication environment connecting them are all stored in the DT server in the MBS. On this basis, a matrix of size M×a is defined To cache MES DD IoV DD of MES at time t IoV It can be expressed as:
[0014]
[0015] Where M is the number of MESs. a represents the number of attributes of each MES. In addition, for the convenience of analysis, the matrix is defined as The first and second columns represent the remaining communication and computing resources of each MES at time t.
[0016]
[0017] As for other information, such as the residual energy of the MES current, it is stored in In the other columns of
[0018] Further, in said S2, a digital twin (DT)-assisted edge collaboration method is proposed to provide offloading services for computationally intensive tasks;
[0019] The dataset storing blockchain information is represented as H is the timestamp set in the block, Established. Represents the timestamp data saved for the jth MES. Z is the version number state set, Established. Indicates the version number stored in the jth MES. where D is the space of all block Merkle roots, is the Merkle root value in the jth MES. Ψ represents the space of transaction information, satisfying The transaction information stored in the jth MES is recorded as
[0020] Suppose the data set of blockchain stored in DT is Define the binary number x j To measure the data consistency of candidate MESs, we have:
[0021]
[0022] If the blockchain message of candidate MESj is consistent with the block data in DT, then the data consistency is verified, that is, x is established.j = 1. Otherwise, x j =0. In order to provide CAV with higher QoE guarantee, DT will pay attention to the channel quality of the communication link while ensuring safety. The better the channel quality between CAV and MESs, the higher the transmission bandwidth that CAV can obtain, thereby reducing data transmission delay. At the same time, under ideal channel conditions, CAV consumes less transmission power when sending the same amount of data. If the channel quality between CAV and MESs is poor, a series of serious problems such as connection interruption and data packet loss will occur. The channel coefficient g between CAV and the jth MES is set to j As the second metric for selecting collaborative MESs. Let g th is the threshold of the channel coefficient. Define a binary number y j To measure the channel quality of the jth candidate communication link between MESj and CAV, we get:
[0023]
[0024] When the link channel coefficient between MESj and CAV is equal to or greater than the threshold, the channel quality is guaranteed and y is established. j =1. Otherwise, it is 0. It is necessary to select MESs that guarantee security and high channel quality from MESs, and it is necessary to satisfy x j =1 and y j =1, we get:
[0025]
[0026] where q j is a binary number, q j =1 means MESj meets the condition, q j =0 means that MESj does not meet the conditions.
[0027] Furthermore, in S3, the delay of executing the system task is analyzed according to the collaborative perception process. When the vehicle chooses to process the perception data locally, the execution delay is as follows:
[0028]
[0029] in The perception time represents the amount of environmental information that needs to be extracted. The data collection delay is not considered in the model because it is as low as a few milliseconds. Indicates the time required to locally process this amount of environmental information. It represents the time required to upload the processed data to MES. Since the scale of the processed results is small, the transmission delay of the result delivery is ignored. It indicates the time when MES uploads the data after local processing of the vehicle to DTDC. The time when these data are uploaded to DTDC can be ignored.
[0030] When the vehicle selects the unloading process, the execution delay is as follows:
[0031]
[0032] Indicates the time required to upload the original data to MES, Indicates the time required for MES to process this amount of environmental information. The execution delay on the RSU side is as follows:
[0033]
[0034] Similarly, we can ignore Indicates the time required for RSU to upload raw data to MES. Indicates the time required for MES to process this amount of environmental information;
[0035] Furthermore, in S4, the optimization problem expression of combining collaborative sensing task allocation, offloading and computing resource allocation to minimize the collaborative sensing task processing delay is:
[0036]
[0037] Where B i The bandwidth resources allocated to vehicle i, β i is the unloading decision of vehicle i, μ i is the proportion of computing resources allocated by MES to vehicle i, μ j is the proportion of computing resources allocated by MES to RSUj;
[0038] Further, in S5, HAS-MADDPG comprises the following steps:
[0039] S61: Initialize the actor network and critic network of each CAV;
[0040] S62: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each CAV, i.e., the initial state of the environment;
[0041] S63: In each time slot, according to the current strategy and state, each CAV selects an action and executes it;
[0042] S64: In each time slot, all CAVs interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool;
[0043] S65: For each CAV, randomly select a small batch of samples from the experience pool;
[0044] S66: For each CAV, calculate the target state value of the critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the actor network;
[0045] S67: Return to S63 after each CAV is completed, otherwise return to S65;
[0046] S68: Return to S62 after completing each time slot, otherwise return to S63;
[0047] S69: Stop after the iteration is completed, otherwise return to S62.
[0048] The beneficial effects of the present invention are as follows: the existing fusion scheme only shares raw data or features, lacks the flexibility to adapt to highly dynamic vehicle network conditions, leads to insufficient or saturated bandwidth or computing resource utilization, and then leads to high overall system latency. In addition, inappropriate resource allocation strategies during the offloading process will lead to extended computing delays, reduced offloading performance, and hinder the safety of autonomous driving. In this paper, we propose a resource allocation scheme in the collaborative perception and edge collaboration process assisted by digital twins, which includes edge collaboration for multi-layer collaborative perception framework and task offloading. First, a multi-layer collaborative perception framework is proposed to help CAV obtain the perception results of the global map. Then, a digital twin (DT)-assisted edge collaboration method is proposed to provide offloading services for computationally intensive tasks. This method allows CAV to select a safe MES with a channel gain that meets the threshold to offload its perception tasks with the assistance of DT to ensure safety and reduce processing delays. Finally, a multi-agent deep deterministic policy gradient (HAS-MADDPG) algorithm based on hybrid action space is proposed for solution. The present invention can optimize the perception resource allocation strategy of collaborative perception and edge collaboration in the Internet of Vehicles and reduce the execution delay in the collaborative perception process.
[0049] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:
[0051] Figure 1 This is a schematic diagram of the physical-virtual scene of the Internet of Vehicles;
[0052] Figure 2 Schematic diagram of collaborative perception process;
[0053] Figure 3 This is the network structure diagram of the HAS-MADDPG algorithm. DETAILED DESCRIPTION
[0054] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.
[0055] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.
[0056] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0057] The present invention proposes a resource allocation scheme for collaborative perception and edge collaboration assisted by digital twins, which can optimize the resource allocation strategy for Internet of Vehicles collaborative and edge collaborative perception and reduce the execution delay of collaborative perception.
[0058] The steps of this method are as follows:
[0059] S1: In the scenario of Internet of Vehicles, a multi-layer collaborative perception global map framework is proposed to help CAV obtain the perception data of the global map. Figure 1 Schematic diagram of the physical-virtual scene of the Internet of Vehicles.
[0060] The IoV scenario consists of a physical IoV system and a digital twin (DT) space. The physical IoV system includes a central macro base station (MBS), several RSUs, MESs, and CAVs. An MES with computing and storage capabilities is deployed near each RSU and is directly connected to the RSU via a wired link. In addition, each CAV and MES is equipped with blockchain technology to assist the CAV in selecting a suitable MES for offloading. The CAV is equipped with various sensors and radars to collect data inside and outside the vehicle, such as engine speed and traffic light signals. When the line of sight is obstructed, the CAV can obtain the latest global map data from the digital twin (DT) system to assist driving. The MBS plays the role of the Digital Twin Data Center (DTDC) in the DT-IoV. In addition to providing traditional base station functions, it also stores all DT data within its coverage area. In addition to being able to access the MBS, the vehicle can also offload its own perception data to the MES for processing through a wireless channel. DT space: Each network unit in the physical entity (including CAV, RSU and MES) sends its current operating status to DT through a real-time channel. VE can be constructed in DT space. IoV Digital simulation models for different service levels. DT-IoV can establish a three-level simulation platform. First, DT collects data generated during CAV driving, such as engine speed, fuel / battery remaining, blockchain data, MES communication and computing resources, blockchain data, communication environment information such as channel state information, etc., to build a single entity level simulation model. On this basis, complex hierarchical models between single entities can be further established. For example, in the vehicle-to-infrastructure (V2I) scenario, the DT space not only stores independent data of CAV and RSU, but also creates a dynamic multi-dimensional and multi-time scale virtual model of the communication environment between vehicles and roadside units, which is very similar to the real environment. Based on these models and big data, the DT space can use machine learning algorithms such as MADDPG to provide intelligent solutions for problems such as task offloading and resource allocation in MES scenarios. As more and more physical entities are built in the DT space, a complex system-level DT IoV that is almost consistent with the real IoV world can be formed. The DT IoV can simulate traffic flow control and guidance strategies on the physical IoV system and intuitively evaluate its effects;
[0061] Assume that there are N vehicles and M RSUs in the system. Since a single MES is coupled to one RSU, the number of MESs is also M. IoVAll information about RSU, MES, CAV and the communication environment connecting them is stored in the DT server in MBS. On this basis, a matrix of size M×a is defined To cache MES DD IoV DD of MES at time t IoV It can be expressed as:
[0062]
[0063] Where M is the number of MESs. a represents the number of attributes of each MES. In addition, for the convenience of analysis, the matrix is defined as The first and second columns represent the remaining communication and computing resources of each MES at time t, which are:
[0064]
[0065] As for other information, such as the residual energy of the MES current, it is stored in in the other columns.
[0066] S2: A digital twin (DT)-assisted edge collaboration approach is proposed to provide offloading services for computationally intensive tasks;
[0067] The dataset storing blockchain information is represented as H is the timestamp set in the block, Established. Represents the timestamp data saved for the jth MES. Z is the version number state set, Established. Indicates the version number stored in the jth MES. where D is the space of all block Merkle roots, is the Merkle root value in the jth MES. Ψ represents the space of transaction information, satisfying The transaction information stored in the jth MES is recorded as
[0068] Suppose the data set of blockchain stored in DT is Define the binary number x j To measure the data consistency of candidate MESs, we have:
[0069]
[0070] If the blockchain message of candidate MESj is consistent with the block data in DT, then the data consistency is verified, that is, x is established. j = 1. Otherwise, x j=0. In order to provide CAV with higher QoE guarantee, DT will pay attention to the channel quality of the communication link while ensuring safety. The better the channel quality between CAV and MESs, the higher the transmission bandwidth that CAV can obtain, thereby reducing data transmission delay. At the same time, under ideal channel conditions, CAV consumes less transmission power when sending the same amount of data. If the channel quality between CAV and MESs is poor, a series of serious problems such as connection interruption and data packet loss will occur. The channel coefficient g between CAV and the jth MES is set to j As the second metric for selecting collaborative MESs. Let g th is the threshold of the channel coefficient. Define a binary number y j To measure the channel quality of the jth candidate communication link between MESj and CAV, we get:
[0071]
[0072] When the link channel coefficient between MESj and CAV is equal to or greater than the threshold, the channel quality is guaranteed and y is established. j =1. Otherwise, it is 0. It is necessary to select MESs that guarantee security and high channel quality from MESs, and it is necessary to satisfy x j =1 and y j =1, we get:
[0073]
[0074] where q j is a binary number, q j =1 means MESj meets the condition, q j =0 means that MESj does not meet the conditions.
[0075] S3: Analyze the delay of system task execution based on the collaborative perception process; Figure 2 A schematic diagram of the collaborative sensing process is shown.
[0076] The execution steps of the scheme are as follows: (1) Task initiation: DT space starts to update the global map, indicating the need for collaborative perception. (2) Decision issuance: DT queries the data of internal MESs, finds MESs that are safe and meet the channel gain requirements through algorithms, and informs CAV. (3) Decision making: As will be discussed in the subsequent algorithm section, the HAS-MADDPG algorithm with centralized training and distributed execution is adopted. CAV and RSU are intelligent agents that make decisions and prepare to execute. (4) Data perception: CAV and RSU use their sensors to perceive the environment and obtain raw data. (5) Data processing: CAV or RSU processes the raw data. The task vehicle and auxiliary vehicle choose whether to calculate locally or offload to MES according to the decision of DTDC, and RSU uses wired transmission to send data to MES. Then they complete their own computing tasks respectively. After the CAV task of local computing is completed, the results must also be uploaded to MES. (6) Processing data upload: Each MES participating in the collaboration uploads the processed data to DTDC. (7) Data fusion: A global map is generated through fusion algorithms or deep learning models. When the CAV is blocked from view, the real-time map model in the DTDC can be downloaded.
[0077] According to the task processing workflow, the overall delay is composed of the following parts: (1) Task initiation time. (2) Decision making time. (3) Decision making time. (4) Data perception time. (5) Data processing time. (6) Processed data upload time. (7) Data fusion time. Usually, the time required for other parts is smaller than the data processing time and can be ignored. This patent mainly analyzes the data processing time.
[0078] Transfer rate It can be expressed as:
[0079]
[0080] Where v k ∈V\v i Indicates that v i Visit the same u j The set of CAVs, p, Denote the transmission power and noise power of CAV respectively, B i Indicates that it is assigned to v i bandwidth, is the channel gain, which is determined by the path loss exponent -α1 and v i with u j The distance between ij Decision. ij The calculation is:
[0081]
[0082] Where d ij v i with u j The distance to the plane, H is the height of RSU.
[0083] When the vehicle chooses to process perception data locally, the execution delay is as follows:
[0084]
[0085] in The perception time represents the amount of environmental information that needs to be extracted. The data collection delay is not considered in the model because it is as low as a few milliseconds. Indicates the time required to locally process this amount of environmental information. It represents the time required to upload the processed data to MES. Since the scale of the processed results is small, the transmission delay of the result delivery is ignored. It indicates the time when MES uploads the data after local processing of the vehicle to DTDC. The time when these data are uploaded to DTDC can be ignored.
[0086] definition is the maximum CPU cycle frequency of CAV (i.e., CPU cycles per second), and the CPU frequency allocated to the local task is satisfy The processing delay of the task of local processing perception data is as follows
[0087]
[0088] Among them, S i Indicates the amount of perceived data, is the CPU frequency used to process perception tasks on the vehicle, and d is the CPU cycles required to process one bit of perception data.
[0089] When the vehicle offloads to the server to process the perception data, the execution delay is as follows:
[0090]
[0091] Indicates the time required to upload the original data to MES, Indicates the time required for MES to process this amount of environmental information. It can be calculated by the following formula:
[0092]
[0093] definition is the highest CPU cycle frequency of the MES connected to RSUj. The processing delay on the MES is expressed as:
[0094]
[0095] where μ i express Assign to v i In summary, the execution delay of unloading vehicles to the server is as follows:
[0096]
[0097] The vehicle selects the solution with the smallest execution delay, so represents the minimum execution delay of the vehicle, that is:
[0098]
[0099] When processing data on the RSU side, the execution delay is as follows:
[0100]
[0101] Similarly, we can ignore and Indicates the time required for RSU to upload raw data to MES. Indicates the time required for MES to process this amount of environmental information:
[0102] It can be calculated by the following formula:
[0103]
[0104] where s j represents the amount of sensing data of RSUj, R wired It represents the wired transmission rate from RSU to MES. The processing delay on MES is expressed as:
[0105]
[0106] where μ j Indicates u j The proportion of computing resources allocated for task processing is assumed to be fixed, because the size of RSU's perception data is relatively stable and will definitely be offloaded to MES. In summary, the execution delay on the RSU side is:
[0107]
[0108] S4: A resource allocation scheme with the goal of minimizing the delay in completing collaborative sensing tasks, constructing a joint problem of sensing task allocation, offloading, and computing resource allocation;
[0109] In order to provide fast, reliable and efficient Internet of Vehicles services and improve safety and user experience, the time required for task completion is crucial. In this section, we formulate the optimization scheme for bandwidth allocation, offloading and computing resource allocation as the problem of minimizing the latency of collaborative sensing task processing. The optimization problem can be formulated as:
[0110]
[0111]
[0112]
[0113]
[0114]
[0115]
[0116] Among them B i Assigned to v i The bandwidth, β i v i Unloading decision, μ i MES for v i The proportion of computing resources allocated. The constraints are described in detail below. C1 The total bandwidth allocated to CAV cannot exceed the total bandwidth of the system. C2 represents v i and u j The total processing time cannot exceed the maximum tolerable delay, C3 means that the total processing time of v i Visit the same u j Vehicles, v i and u j The sum of the proportions of computing resources obtained is not greater than 1. C4 indicates that the MESs selected by CAV for task offloading must meet two conditions: first, the MESs blockchain message is consistent with the block data in DT, verifying data consistency to ensure data security, and second, the link channel coefficient between CAV and MESs is equal to or greater than the threshold to provide guarantee for communication quality. C5 indicates the restrictions on CPU cycle frequency by CAV and MESs.
[0117] S5: To solve the optimization problem, a multi-agent deep deterministic policy gradient algorithm HAS-MADDPG in a hybrid action space is proposed to learn the resource allocation scheme with the minimum task completion delay; Figure 3 This is the network structure diagram of the HAS-MADDPG algorithm.
[0118] The optimization problem is transformed into a Markov decision process, and the state space is set as:
[0119]
[0120] They represent the remaining computing resource state of MES, the channel gain of the communication link between CAV and MES, and the connection state between CAV and MES. Action space: Based on the offloading decision β i Discrete actions and bandwidth resource allocation B i , calculate resource allocation ratio μ i The continuous variables are used together as the mixed action a of CAV i ={β i ,B i ,μ i}∈A.
[0121] award: Where β is a constant, which is to increase the reward when the delay decreases.
[0122] The goal of training the value network is to minimize the prediction error of the state-action value function. Assume that the parameters of each CAV are θ = {θ1, θ2, ..., θ N}. For each CAVi, use the target network to calculate the target value. Calculate the target action a' n , where a' n =μ' n (s') indicates the next state of the nth agent s' n The target action under , n∈{1,2,...,N}. The target value is
[0123] y j =r i +γQ′ i (s′,a i )
[0124] Where Q' i is the target value network of CAVi, and γ is the discount factor. The value network parameters θ are updated by minimizing the mean square error loss. i :
[0125]
[0126] Where B is a batch extracted from the experience replay area. When training the policy network, assume that the policy network parameters of each CAV are ω={ω1,ω2,...,ω N}. For each CAV i, the policy network is updated using the policy gradient. The policy gradient is:
[0127]
[0128] Update the policy network parameters ω using the gradient ascent method i For the target network parameters of the value network and the policy network, soft updates are used:
[0129] θ′ i =τθ′ i +(1-τ)θ i
[0130] ω′ i =τω′ i +(1-τ)ω i
[0131] The HAS-MADDPG algorithm specifically includes the following steps:
[0132] S61: Initialize the actor network and critic network of each CAV;
[0133] S62: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each CAV, i.e., the initial state of the environment;
[0134] S63: In each time slot, according to the current strategy and state, each CAV selects an action and executes it;
[0135] S64: In each time slot, all CAVs interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool;
[0136] S65: For each CAV, randomly select a small batch of samples from the experience pool;
[0137] S66: For each CAV, calculate the target state value of the critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the actor network;
[0138] S67: After completing each CAV, soft update the target network parameters of the value network and the strategy network, and then return to S63, otherwise return to S65;
[0139] S68: Return to S62 after completing each time slot, otherwise return to S63;
[0140] S69: Stop after the iteration is completed, otherwise return to S62.
[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.
Claims
1. A digital twin-assisted collaborative perception and edge collaboration resource allocation method, characterized by: The method comprises the following steps: S1: In the scenario of Internet of Vehicles, a multi-layer collaborative perception global map framework is constructed to help connected automatic vehicles (CAV) obtain the perception results of the global map; S2: Using the Digital Twin (DT) space to assist the networked autonomous vehicle (CAV) in selecting a safe edge computing server (Mobile Edge Server, MES) with a channel gain that meets the threshold to offload the perception task; S3: Analyze the delay of system task execution based on the collaborative perception process; S4: With the goal of minimizing the delay in completing the collaborative sensing task, a joint optimization problem of bandwidth allocation, offloading ratio, and computing resource allocation is constructed; S5: The multi-agent deep deterministic policy gradient algorithm HAS-MADDPG in hybrid action space is used to solve the optimization problem and learn the resource allocation plan with the minimum task completion delay.
2. The digital twin-assisted collaborative perception and edge collaboration resource allocation method according to claim 1 is characterized by: In S1, the Internet of Vehicles scenario consists of a physical Internet of Vehicles system and a digital twin (DT) space; The physical Internet of Vehicles system includes a central macro base station (MBS), several roadside units (RSUs), edge computing servers (MESs), and networked automated vehicles (CAVs); an MES with computing and storage capabilities is deployed next to each RSU and is directly connected to the RSU through a wired link; each CAV and MES is equipped with blockchain technology to assist the CAV in selecting the appropriate MES for unloading; the CAV is equipped with various sensors and radars to collect data inside and outside the vehicle; when the line of sight is obstructed, the CAV obtains the latest global map data from the digital twin DT system to assist driving; the MBS plays the role of the Digital Twin Data Center (DTDC) in the Digital Twin-Internet of Vehicles (DT-IoV), storing all DT data within its coverage area; in addition to being able to access the MBS, the vehicle unloads its own perception data to the MES for processing through a wireless channel; DT space: each network unit in the physical entity includes CAV, RSU, and MES, and each network unit sends its current operating status to the DT through a real-time channel, building a VE in the DT space IoV Digital simulation models with different service levels; DT-IoV establishes a three-level simulation platform; First, DT collects data generated during CAV driving, including engine speed, remaining fuel, remaining battery, blockchain data, communication and computing resources of MES, blockchain data and communication environment information, and builds a single entity level simulation model; Establishes a complex hierarchical model between single entities; In the vehicle-to-infrastructure (V2I) scenario, the DT space stores independent data of CAV and RSU, and creates a dynamic multi-dimensional and multi-time scale communication environment virtual model between vehicles and roadside units; DT space uses machine learning algorithms to provide intelligent solutions for task offloading and resource allocation problems in MES scenarios; As physical entities are built in the DT space, a DT vehicle network is formed; The DT vehicle network simulates traffic flow control and guidance strategies on the physical vehicle network system and intuitively evaluates its effects; Assume that there are N vehicles and M RSUs in the system, a single MES is coupled with one RSU, and the number of MESs is M; RSU, MES, CAV and the communication environment connecting them are all stored in the DT server in MBS; define a matrix of size M×a To cache the Digital Data of Internet of Vehicles (DD IoV ); DD of MES at time t IoV It is expressed as: Where M is the number of MES; a represents the number of attributes of each MES; represents the matrix of MES at time t; Defining the Matrix The first and second columns represent the remaining communication and computing resources of each MES at time t, which are: The residual energy of the MES current is stored in in the other columns.
3. The digital twin-assisted collaborative perception and edge collaboration resource allocation method according to claim 1, characterized in that: In S2, a digital twin DT-assisted edge collaboration method is proposed to provide offloading services for computationally intensive tasks; The dataset storing blockchain information is represented as H is the timestamp set in the block, Established; represents the timestamp data saved by the jth MES; Z is the version number state set, Established; Indicates the version number stored in the jth MES; where D is the space of all block Merkle roots, is the Merkle root value in the jth MES; Ψ represents the space of transaction information, satisfying The transaction information stored in the jth MES is recorded as Suppose the data set of blockchain stored in DT is Define the binary number x j To measure the data consistency of candidate MESs, there are: If the blockchain message of candidate MESj is consistent with the block data in DT, then the data consistency is verified, that is, x is established. j =1; otherwise, x j =0; the better the channel quality between CAV and MESs, the higher the transmission bandwidth obtained by CAV; when sending the same amount of data, CAV consumes less transmission power; the channel coefficient g between CAV and the jth MES is j As the second metric for selecting collaborative MESs; let g th is the threshold of the channel coefficient; Define a binary number y j To measure the channel quality of the jth candidate communication link between MESj and CAV, we get: When the link channel coefficient between MESj and CAV is equal to or greater than the threshold, the channel quality is guaranteed and y is established. j =1; otherwise, y j = 0; select MESs that guarantee security and have high channel quality from MESs, which needs to satisfy x j =1 and y j =1, we get: where q j is a binary number, q j =1 means that MES j meets the condition, q j =0 means that MES j does not meet the condition.
4. The digital twin-assisted collaborative perception and edge collaboration resource allocation method according to claim 1, characterized in that: In S3, the delay of executing the system tasks is analyzed according to the collaborative perception process; when the vehicle chooses to process the perception data locally, the execution delay is as follows: in, represents the perception time of the amount of environmental information that needs to be extracted, without considering the data collection delay, Indicates the time required to locally process this amount of environmental information. It represents the time required to upload the processed data to MES, ignoring the transmission delay of the result delivery; Indicates the time when MES uploads the data after local processing of the vehicle to DTDC, ignoring the time when the data is uploaded to DTDC; When the vehicle selects the unloading process, the execution delay is as follows: Indicates the time required to upload the original data to MES, It indicates the time required by MES to process this amount of environmental information. The vehicle selects the solution with the smallest execution delay, so represents the minimum execution delay of the vehicle, that is: Execution delay on the RSU side It is expressed as follows: Ignore the same Indicates the time required for RSU to upload raw data to MES. Indicates the time required for MES to process this amount of environmental information.
5. The digital twin-assisted collaborative perception and edge collaboration resource allocation method according to claim 1, characterized in that: In S4, the optimization problem expression of combining collaborative sensing task allocation, offloading and computing resource allocation to minimize the collaborative sensing task processing delay is: Where B i The bandwidth resources allocated to vehicle i, β i is the unloading decision of vehicle i, μ i is the proportion of computing resources allocated by MES to vehicle i, μ j is the proportion of computing resources allocated by MES to RSUj.
6. The digital twin-assisted collaborative perception and edge collaboration resource allocation method according to claim 1, characterized in that: In S5, HAS-MADDPG comprises the following steps: S61: Initialize the actor network and critic network of each CAV; S62: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each CAV, i.e., the initial state of the environment; S63: In each time slot, according to the current strategy and state, each CAV selects an action and executes it; S64: In each time slot, all CAVs interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool; S65: For each CAV, randomly select a small batch of samples from the experience pool; S66: For each CAV, calculate the target state value of the critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the actor network; S67: Return to S63 after each CAV is completed, otherwise return to S65; S68: Return to S62 after completing each time slot, otherwise return to S63; S69: Stop after the iteration is completed, otherwise return to S62.
Citation Information
Patent Citations
Calculation unloading and resource allocation method for vehicle-road cooperation system
CN116896561A
DTS deployment and resource allocation method based on edge cloud cooperation
CN117614949A
Digital twinning enabled industrial internet of things resource allocation system and method
CN117749635A
MADFPG-based task unloading and resource allocation joint optimization method
CN117880122A
Digital twinning-based car networking task unloading method
CN118075265A
Cited By
Resource allocation method and system in communication, inductance and calculation integrated Internet of Vehicles system
CN119562356A
Networked autonomous vehicle communication and cooperative control method for mixed traffic flow
CN119964387A
BEV-based multi-source sensing data fusion and vehicle and road resource allocation method
CN120708406A
A method for multi-source perception data fusion and road-vehicle resource allocation based on BEV
CN120708406B
Resource allocation method and system for 5G communication in underground coal mine
CN121174295A
