Digital twin assisted collaborative perception and edge collaboration resource allocation method

Abstract

The present application relates to a kind of digital twin assisted collaborative perception and edge collaboration resource allocation method, belong to mobile communication technical field.The method includes the following steps: S1: in the Internet of Vehicles scene, a kind of multi-layer collaborative perception global map framework is proposed, help connected automated vehicle CAV obtain the perception result of global map;S2: a kind of digital twin (DT) assisted edge collaboration method is proposed to provide offload service for computationally intensive tasks;S3: according to collaborative perception process analysis system task execution delay;S4: with the minimum collaborative perception task completion delay as the goal of resource allocation scheme, construct the joint problem of bandwidth allocation, unloading and computing resource allocation;S5: propose the multi-agent deep deterministic policy gradient HAS-MADDPG algorithm of mixed action space to solve optimization problem.The present application can realize the optimization of resource allocation strategy for Internet of Vehicles collaborative perception and edge offload, reduce the execution delay of collaborative perception.

Description

Technical Field

[0001] The present invention belongs to the field of mobile communication technology and relates to a resource allocation method for collaborative perception and edge collaboration assisted by digital twins. Background Art

[0002] The perception capabilities of individual connected automated vehicles (CAVs) have inherent limitations. The emergence of collaborative perception technology in connected vehicles (IoVs) can effectively overcome this problem. By fusing perception data from multiple CAVs and RSUs, the perception range can be significantly expanded and accuracy improved. However, existing single-layer collaborative perception solutions still face several challenges. The first is inefficient resource utilization. In the early fusion phase, excessive raw data transmission can lead to bandwidth saturation and underutilization of computing power. In the late fusion phase, sending only processed object data can lead to computing power saturation and bandwidth underutilization. Even in the mid-term fusion phase, resource utilization efficiency is often low under dynamic network conditions. Secondly, system performance is poor. This imbalance in resource utilization increases overall system latency, impacting the real-time and safety of collaborative perception. Therefore, a more flexible and efficient collaborative perception solution is needed to improve system performance. Therefore, there is an urgent need to develop an IoV collaborative perception framework that can dynamically adapt to network conditions and fully utilize network resources to meet the needs of applications such as autonomous driving.

[0003] Due to the limited computing power of CAVs, they often choose early fusion when sensing data and offload it to the MES for processing. However, with the explosive growth of data volumes, a single MES struggles to cope with multiple tasks, necessitating a new solution. Edge collaboration compensates for the limitations of a single MES in communication, computing, and storage resources by distributing tasks, saving system power and time. However, as the number of collaborating nodes increases, security becomes a primary concern for CAVs. Some nodes may be attacked or intentionally configured as malicious, leading to information leakage or tampering. Artificial intelligence (AI) has achieved significant success in fields such as intelligent transportation, and many studies are applying AI to address resource allocation and task offloading in mobile edge computing (MEC). However, limited storage and computing resources in CAVs result in insufficient data for AI model training, resulting in inaccurate predictions. Using DT to train AI models in an edge collaboration model can assist CAVs in selecting fusion methods and performing task offloading, improving overall network performance. Therefore, using DT to explore the application of AI in edge node collaboration to help CAVs intelligently offload tasks is an issue worthy of in-depth research, revealing the necessity of exploring new offloading methods. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a resource allocation method for collaborative perception and edge collaboration assisted by digital twins, which can optimize the resource allocation strategy of collaborative perception and edge collaboration in the Internet of Vehicles and reduce the execution delay of collaborative perception.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A resource allocation method for collaborative perception and edge collaboration in a digital twin-assisted Internet of Vehicles (IoV) includes the following steps:

[0007] S1: In the context of the Internet of Vehicles (IoV), a multi-layer collaborative perception global map framework is proposed to help connected autonomous vehicles (CAVs) obtain global map perception results;

[0008] S2: A digital twin (DT)-assisted edge collaboration method is proposed to provide offloading services for computationally intensive tasks;

[0009] S3: Analyze the delay of system task execution based on the collaborative perception process;

[0010] S4: A resource allocation scheme with the goal of minimizing the delay in completing collaborative sensing tasks, constructing a joint problem of sensing task allocation, offloading, and computing resource allocation;

[0011] S5: A multi-agent deep deterministic policy gradient HAS-MADDPG algorithm in a hybrid action space is proposed to solve the optimization problem and learn the resource allocation solution with the minimum task completion delay.

[0012] Furthermore, in S1, the IoV scenario consists of a physical IoV system and a digital twin (DT) space. The physical IoV system includes a central macro base station (MBS), several RSUs, MESs, and CAVs. An MES with computing and storage capabilities is deployed near each RSU and is directly connected to the RSU via a wired link. Furthermore, each CAV and MES is equipped with blockchain technology, which helps the CAV select the appropriate MES for offloading. The CAV is equipped with various sensors and radars to collect data from inside and outside the vehicle, such as engine speed and traffic light signals. When vision is obstructed, the CAV can obtain the latest global map data from the digital twin (DT) system to assist driving. The MBS serves as the Digital Twin Data Center (DTDC) in the DT-IoV. In addition to providing traditional base station functions, it also stores all DT data within its coverage area. In addition to being able to access the MBS, vehicles can also offload their own sensory data to the MES for processing via wireless channels. DT space: Each network unit in the physical entity (including CAV, RSU and MES) sends its current operating status to DT through a real-time channel. VE can be constructed in DT space. IoV Digital simulation models for different service levels. DT-IoV can establish a three-level simulation platform. First, DT collects data generated by CAVs during driving, such as engine speed, fuel / battery remaining levels, blockchain data, MES communication and computing resources, blockchain data, and communication environment information such as channel state information, to build a single-entity-level simulation model. This foundation can be further used to build complex hierarchical models between single entities. For example, in vehicle-to-infrastructure (V2I) scenarios, the DT space not only stores independent data from CAVs and RSUs but also creates a dynamic, multi-dimensional, and multi-timescale virtual model of the communication environment between vehicles and roadside units, closely resembling the real environment. Based on these models and big data, the DT space can use machine learning algorithms such as MADDPG to provide intelligent solutions for problems such as task offloading and resource allocation in MES scenarios. As more and more physical entities are built within the DT space, a complex system-level DT IoV that is nearly identical to the real IoV world can be formed. The DT IoV can simulate traffic flow control and guidance strategies on the physical IoV system and intuitively evaluate their effectiveness.

[0013] Assume that there are N vehicles and M RSUs in the system. Since a single MES is coupled to one RSU, the number of MESs is also M. RSU, MES, CAV and the communication environment connecting them are all stored in the DT server in the MBS. Based on this, a matrix of size M×a is defined To cache MES DD IoV DD of MES at time t IoV It can be expressed as:

[0014]

[0015] Where M is the number of MESs. a represents the number of attributes of each MES. In addition, for the convenience of analysis, the matrix is ​​defined as The first and second columns represent the remaining communication and computing resources of each MES at time t.

[0016]

[0017] As for other information, such as the residual energy of the MES current, it is stored in In the other columns;

[0018] Furthermore, in said S2, a digital twin (DT)-assisted edge collaboration method is proposed to provide offloading services for computationally intensive tasks;

[0019] The dataset storing blockchain information is represented as H is the timestamp set in the block, Established. Represents the timestamp data saved for the jth MES. Z is the version number state set, Established. Indicates the version number stored in the j-th MES. where D is the space of all block Merkle roots, is the Merkle root value in the jth MES. Ψ represents the space of transaction information, satisfying The transaction information stored in the jth MES is recorded as

[0020] Assume that the data set of the blockchain stored in DT is Define the binary number x j To measure the data consistency of candidate MESs, we have:

[0021]

[0022] If the blockchain message of candidate MESj is consistent with the block data in DT, then the data consistency is verified, that is, x is established.j = 1. Otherwise, x j =0. To provide higher QoE guarantees for CAVs, DT will focus on the channel quality of the communication link while ensuring safety. The better the channel quality between CAVs and MESs, the higher the transmission bandwidth that CAVs can obtain, thereby reducing data transmission delays. At the same time, under ideal channel conditions, CAVs consume less transmission power when sending the same amount of data. If the channel quality between CAVs and MESs is poor, a series of serious problems such as connection interruption and packet loss will occur. The channel coefficient g between CAV and the jth MES is calculated. j As the second metric for selecting collaborative MESs. Let g th is the threshold of the channel coefficient. Define a binary number y j To measure the channel quality of the j-th candidate communication link between MESj and CAV, we get:

[0023]

[0024] When the link channel coefficient between MESj and CAV is equal to or greater than the threshold, the channel quality is guaranteed and y is established. j =1. Otherwise, it is 0. It is necessary to select MESs that guarantee security and high channel quality from MESs, and it is necessary to satisfy x j =1 and y j =1, we get:

[0025]

[0026] where q j is a binary number, q j =1 means MESj meets the conditions, q j =0 means that MESj does not meet the conditions.

[0027] Furthermore, in S3, the delay of system task execution is analyzed according to the collaborative perception process. When the vehicle chooses to process the perception data locally, the execution delay is as follows:

[0028]

[0029] in The perception time represents the amount of environmental information that needs to be extracted. The data collection delay is not considered in the model because it is as low as a few milliseconds. Indicates the time required for local processing of this amount of environmental information, It represents the time required to upload the processed data to MES. Since the scale of the processed results is small, the transmission delay of the results is ignored. Indicates the time when MES uploads the data after local vehicle processing to DTDC. The time when this data is uploaded to DTDC can be ignored.

[0030] When the vehicle selects the unloading process, the execution delay is as follows:

[0031]

[0032] Indicates the time required to upload the original data to MES, Indicates the time required for MES to process this amount of environmental information. The execution delay on the RSU side is as follows:

[0033]

[0034] Similarly, you can ignore Indicates the time required for RSU to upload raw data to MES, Indicates the time required for MES to process this amount of environmental information;

[0035] Furthermore, in S4, the optimization problem expression for minimizing the collaborative sensing task processing delay by combining collaborative sensing task allocation, offloading, and computing resource allocation is:

[0036]

[0037] Where B i The bandwidth resources allocated to vehicle i, β i is the unloading decision of vehicle i, μ i is the proportion of computing resources allocated by MES to vehicle i, μ j is the proportion of computing resources allocated by MES to RSUj;

[0038] Furthermore, in said S5, HAS-MADDPG comprises the following steps:

[0039] S61: Initialize the actor network and critic network of each CAV;

[0040] S62: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each CAV, i.e., the initial state of the environment;

[0041] S63: At each time slot, based on the current strategy and state, each CAV selects an action and executes it;

[0042] S64: In each time slot, all CAVs interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool;

[0043] S65: For each CAV, randomly sample a small batch of samples from the experience pool;

[0044] S66: For each CAV, calculate the target state value of the critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the actor network;

[0045] S67: Return to S63 after completing each CAV, otherwise return to S65;

[0046] S68: Return to S62 after completing each time slot, otherwise return to S63;

[0047] S69: Stop after the iteration is completed, otherwise return to S62.

[0048] The present invention offers the following advantages: Existing fusion solutions only share raw data or features, lacking the flexibility to adapt to highly dynamic vehicle network conditions. This leads to insufficient or saturated bandwidth or computing resource utilization, resulting in high overall system latency. Furthermore, during offloading, inappropriate resource allocation strategies can extend computational latency, reduce offloading performance, and hinder the safety of autonomous driving. In this paper, we propose a resource allocation scheme for collaborative perception and edge collaboration assisted by digital twins. This scheme incorporates a multi-layer collaborative perception framework and edge collaboration for task offloading. First, a multi-layer collaborative perception framework is proposed to help CAVs obtain global map perception results. Then, a digital twin (DT)-assisted edge collaboration method is proposed to provide offloading services for computationally intensive tasks. This method, with the assistance of the DT, allows CAVs to select safe MESs with channel gain that meets a threshold to offload their perception tasks, ensuring safety and reducing processing latency. Finally, a hybrid action space-based multi-agent deep deterministic policy gradient (HAS-MADDPG) algorithm is proposed for solution. This invention optimizes the perception resource allocation strategy for collaborative perception and edge collaboration in connected vehicles, reducing execution latency during collaborative perception.

[0049] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0051] Figure 1 A schematic diagram of the physical-virtual scenario of the Internet of Vehicles;

[0052] Figure 2 Schematic diagram of collaborative perception process;

[0053] Figure 3 This is the network structure diagram of the HAS-MADDPG algorithm. DETAILED DESCRIPTION

[0054] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0055] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0056] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0057] The present invention proposes a resource allocation scheme for collaborative perception and edge collaboration assisted by digital twins, which can optimize the resource allocation strategy of Internet of Vehicles collaboration and edge collaboration perception and reduce the execution delay of collaborative perception.

[0058] The steps of this method are as follows:

[0059] S1: In the context of the Internet of Vehicles, a multi-layer collaborative perception global map framework is proposed to help CAVs obtain perception data of the global map. Figure 1 Schematic diagram of the physical-virtual scene of the Internet of Vehicles.

[0060] The IoV scenario consists of a physical IoV system and a digital twin (DT) space. The physical IoV system includes a central macro base station (MBS), several RSUs, MESs, and CAVs. An MES with computing and storage capabilities is deployed near each RSU and is directly connected to the RSU via a wired link. Furthermore, each CAV and MES is equipped with blockchain technology, which helps the CAV select the appropriate MES for offloading. CAVs are equipped with various sensors and radars to collect data from both inside and outside the vehicle, such as engine speed and traffic light signals. When visibility is obstructed, CAVs can obtain the latest global map data from the digital twin (DT) system to assist driving. The MBS serves as the digital twin data center (DTDC) in the DT-IoV. In addition to providing traditional base station functions, it also stores all DT data within its coverage area. In addition to accessing the MBS, vehicles can also offload their own sensory data to the MES for processing via wireless channels. DT space: Each network unit in the physical entity (including CAV, RSU and MES) sends its current operating status to DT through a real-time channel. VE can be constructed in the DT space. IoV Digital simulation models for different service levels. DT-IoV can establish a three-level simulation platform. First, DT collects data generated by CAVs during driving, such as engine speed, fuel / battery remaining levels, blockchain data, MES communication and computing resources, blockchain data, and communication environment information such as channel state information, to build a single-entity-level simulation model. This foundation can be further used to build complex hierarchical models between single entities. For example, in vehicle-to-infrastructure (V2I) scenarios, the DT space not only stores independent data from CAVs and RSUs but also creates a dynamic, multi-dimensional, and multi-timescale virtual model of the communication environment between vehicles and roadside units, closely resembling the real environment. Based on these models and big data, the DT space can use machine learning algorithms such as MADDPG to provide intelligent solutions for problems such as task offloading and resource allocation in MES scenarios. As more and more physical entities are built within the DT space, a complex system-level DT IoV that is nearly identical to the real IoV world can be formed. The DT IoV can simulate traffic flow control and guidance strategies on the physical IoV system and intuitively evaluate their effectiveness.

[0061] Assume that there are N vehicles and M RSUs in the system. Since a single MES is coupled with one RSU, the number of MESs is also M. IoVAll information about RSU, MES, CAV and the communication environment connecting them is stored in the DT server in MBS. On this basis, a matrix of size M×a is defined To cache MES DD IoV DD of MES at time t IoV It can be expressed as:

[0062]

[0063] Where M is the number of MESs. a represents the number of attributes of each MES. In addition, for the convenience of analysis, the matrix is ​​defined as The first and second columns represent the remaining communication and computing resources of each MES at time t, which are:

[0064]

[0065] As for other information, such as the residual energy of the MES current, it is stored in in the other columns.

[0066] S2: A digital twin (DT)-assisted edge collaboration method is proposed to provide offloading services for computationally intensive tasks;

[0067] The dataset storing blockchain information is represented as H is the timestamp set in the block, Established. Represents the timestamp data saved for the jth MES. Z is the version number state set, Established. Indicates the version number stored in the j-th MES. where D is the space of all block Merkle roots, is the Merkle root value in the jth MES. Ψ represents the space of transaction information, satisfying The transaction information stored in the jth MES is recorded as

[0068] Assume that the data set of the blockchain stored in DT is Define the binary number x j To measure the data consistency of candidate MESs, we have:

[0069]

[0070] If the blockchain message of candidate MESj is consistent with the block data in DT, then the data consistency is verified, that is, x is established. j = 1. Otherwise, x j=0. To provide higher QoE guarantees for CAVs, DT will focus on the channel quality of the communication link while ensuring safety. The better the channel quality between CAVs and MESs, the higher the transmission bandwidth that CAVs can obtain, thereby reducing data transmission delays. At the same time, under ideal channel conditions, CAVs consume less transmission power when sending the same amount of data. If the channel quality between CAVs and MESs is poor, a series of serious problems such as connection interruption and packet loss will occur. The channel coefficient g between CAV and the jth MES is calculated. j As the second metric for selecting collaborative MESs. Let g th is the threshold of the channel coefficient. Define a binary number y j To measure the channel quality of the j-th candidate communication link between MESj and CAV, we get:

[0071]

[0072] When the link channel coefficient between MESj and CAV is equal to or greater than the threshold, the channel quality is guaranteed and y is established. j =1. Otherwise, it is 0. It is necessary to select MESs that guarantee security and high channel quality from MESs, and it is necessary to satisfy x j =1 and y j =1, we get:

[0073]

[0074] where q j is a binary number, q j =1 means MESj meets the conditions, q j =0 means that MESj does not meet the conditions.

[0075] S3: Analyze the delay of system task execution based on the collaborative perception process; Figure 2 A schematic diagram of the collaborative perception process is shown.

[0076] The execution steps of the scheme are as follows: (1) Task initiation: DT space starts to update the global map, indicating the need for collaborative perception. (2) Decision issuance: DT queries the data of internal MESs, finds MESs that are safe and meet the channel gain requirements through the algorithm, and informs CAV. (3) Decision making: As will be discussed in the subsequent algorithm section, the HAS-MADDPG algorithm with centralized training and distributed execution is adopted. CAV and RSU act as intelligent agents to make decisions and prepare to execute. (4) Data perception: CAV and RSU use their sensors to perceive the environment and obtain raw data. (5) Data processing: CAV or RSU processes the raw data. The task vehicle and auxiliary vehicle choose whether to perform local calculations or offload to MES according to the decision of DTDC, while RSU uses wired transmission to send data to MES. Then they complete their own computing tasks respectively. After the CAV task of local calculation is completed, the results must also be uploaded to MES. (6) Processing data upload: Each MES participating in the collaboration uploads the processed data to DTDC. (7) Data fusion: Generate a global map through fusion algorithms or deep learning models. When the CAV is blocked from view, it can download the real-time map model in the DTDC.

[0077] According to the task processing workflow, the overall latency is composed of the following components: (1) Task initiation time. (2) Decision-making time. (3) Decision-making time. (4) Data perception time. (5) Data processing time. (6) Processed data upload time. (7) Data fusion time. The time required for other components is usually small compared to the data processing time and can be ignored. This patent focuses on analyzing the data processing time.

[0078] Transfer rate It can be expressed as:

[0079]

[0080] Where v k ∈V\v i Indicates that v i Access the same u j The set of CAVs, p, Denote the transmission power and noise power of CAV respectively, B i Indicates the assignment to v i bandwidth, is the channel gain, which is determined by the path loss exponent -α1 and v i with u j The distance d ij Decision. ij The calculation is:

[0081]

[0082] Where d ij v i with u j The distance to the plane, H is the height of RSU.

[0083] When the vehicle chooses to process perception data locally, the execution delay is as follows:

[0084]

[0085] in The perception time represents the amount of environmental information that needs to be extracted. The data collection delay is not considered in the model because it is as low as a few milliseconds. Indicates the time required for local processing of this amount of environmental information, It represents the time required to upload the processed data to MES. Since the scale of the processed results is small, the transmission delay of the results is ignored. Indicates the time when MES uploads the data after local vehicle processing to DTDC. The time when this data is uploaded to DTDC can be ignored.

[0086] definition is the highest CPU cycle frequency of CAV (i.e., CPU cycles per second), and the CPU frequency allocated to the local task is satisfy The processing delay of the task of local processing perception data is as follows

[0087]

[0088] Among them, S i Indicates the amount of perceived data, is the CPU frequency used to process perception tasks on the vehicle, and d is the CPU cycles required to process one bit of perception data.

[0089] When the vehicle offloads to the server to process the perception data, the execution delay is as follows:

[0090]

[0091] Indicates the time required to upload the original data to MES, Indicates the time required for MES to process this amount of environmental information. It can be calculated by the following formula:

[0092]

[0093] definition is the maximum CPU cycle frequency of the MES connected to RSUj. The processing delay on the MES is expressed as:

[0094]

[0095] where μ i express Assigned to v i In summary, the execution delay of unloading the vehicle to the server is as follows:

[0096]

[0097] The vehicle selects the solution with the smallest execution delay, so represents the minimum execution delay of the vehicle, that is:

[0098]

[0099] When processing data on the RSU side, the execution delay is as follows:

[0100]

[0101] Similarly, you can ignore and Indicates the time required for RSU to upload raw data to MES, Indicates the time required for MES to process this amount of environmental information:

[0102] It can be calculated by the following formula:

[0103]

[0104] where s j represents the amount of perception data of RSUj, R wired It represents the wired transmission rate from RSU to MES. The processing delay on MES is expressed as:

[0105]

[0106] where μ j Indicates u j The proportion of computing resources allocated for task processing is assumed to be fixed, because the size of RSU's perception data is relatively stable and will definitely be offloaded to the MES. In summary, the execution delay on the RSU side is:

[0107]

[0108] S4: A resource allocation scheme with the goal of minimizing the delay in completing collaborative sensing tasks, constructing a joint problem of sensing task allocation, offloading, and computing resource allocation;

[0109] To provide fast, reliable, and efficient IoV services and improve safety and user experience, the time required for task completion is crucial. In this section, we formulate the optimization scheme for bandwidth allocation, offloading, and computing resource allocation as a problem to minimize the processing latency of collaborative sensing tasks. The optimization problem can be formulated as:

[0110]

[0111]

[0112]

[0113]

[0114]

[0115]

[0116] Among them B i Assigned to v i Bandwidth, β i v i Offloading decision, μ i MES for v i The proportion of computing resources allocated. The constraints are detailed below. C1 The total bandwidth allocated to CAV cannot exceed the total bandwidth of the system. C2 represents v i and u j The total processing time cannot exceed the maximum tolerable delay, C3 means that the total processing time of v i Access the same u j Vehicles, v i and u j The sum of the proportions of computing resources obtained must not exceed 1. C4 indicates that the MESs selected by the CAV for task offloading must meet two conditions: first, the MESs blockchain message is consistent with the block data in the DT, verifying data consistency to ensure data security; second, the link channel coefficient between the CAV and MESs is equal to or greater than the threshold to ensure communication quality. C5 represents the CPU cycle frequency restrictions imposed by the CAV and MESs.

[0117] S5: To solve the optimization problem, we propose a multi-agent deep deterministic policy gradient algorithm (HAS-MADDPG) in a hybrid action space to learn a resource allocation solution that minimizes task completion delay. Figure 3 This is the network structure diagram of the HAS-MADDPG algorithm.

[0118] The optimization problem is transformed into a Markov decision process, and the state space is set as:

[0119]

[0120] Respectively represent the remaining computing resource state of MES, the channel gain of the communication link between CAV and MES, and the connection state between CAV and MES. Action space: with offloading decision β i Discrete actions and bandwidth resource allocation B i , computing resource allocation ratio μ i The continuous variables are used together as the mixed action a of CAV i ={β i ,B i ,μ i}∈A.

[0121] award: Where β is a constant, which is used to increase the reward when the delay decreases.

[0122] The goal of training the value network is to minimize the prediction error of the state-action value function. Assume that the parameters of each CAV are θ={θ1,θ2,...,θ N}. For each CAVi, use the target network to calculate the target value. Calculate the target action a' n , where a' n =μ' n (s') represents the next state s' of the nth agent n The target action under , n∈{1,2,...,N}. The target value is

[0123] y j =r i +γQ′ i (s′,a i )

[0124] where Q' i is the target value network of CAVi, and γ is the discount factor. The value network parameters θ are updated by minimizing the mean square error loss. i :

[0125]

[0126] Where B is a batch extracted from the experience replay area. When training the policy network, assume that the policy network parameters of each CAV are ω={ω1,ω2,...,ω N For each CAV i, the policy network is updated using the policy gradient. The policy gradient is:

[0127]

[0128] Update the policy network parameters ω using the gradient ascent method i For the target network parameters of the value network and the policy network, soft updates are used:

[0129] θ′ i =τθ′ i +(1-τ)θ i

[0130] ω′ i =τω′ i +(1-τ)ω i

[0131] The HAS-MADDPG algorithm specifically includes the following steps:

[0132] S61: Initialize the actor network and critic network of each CAV;

[0133] S62: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each CAV, i.e., the initial state of the environment;

[0134] S63: At each time slot, based on the current strategy and state, each CAV selects an action and executes it;

[0135] S64: At each time slot, all CAVs interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool;

[0136] S65: For each CAV, randomly sample a small batch of samples from the experience pool;

[0137] S66: For each CAV, calculate the target state value of the critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the actor network;

[0138] S67: After completing each CAV, soft-update the target network parameters of the value network and the policy network, and then return to S63, otherwise return to S65;

[0139] S68: Return to S62 after completing each time slot, otherwise return to S63;

[0140] S69: Stop after the iteration is completed, otherwise return to S62.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A digital twin-assisted collaborative perception and edge collaboration resource allocation method, characterized by: The method comprises the following steps: S1: In the connected vehicle scenario, a multi-layer collaborative perception global map framework is constructed to help connected autonomous vehicles (CAVs) obtain global map perception results; S2: Using digital twin DT space to assist networked autonomous vehicles (CAVs) in selecting edge computing servers (MESs) that are safe and have channel gain that meets the threshold for perception task offloading. S3: Analyze the delay of system task execution according to the collaborative perception process; In S3, the delay of system task execution is analyzed according to the collaborative perception process; when the vehicle chooses to process the perception data locally, the execution delay is as follows: in, represents the perception time of the amount of environmental information that needs to be extracted, without considering the data collection delay, Indicates the time required for local processing of this amount of environmental information, It represents the time required to upload the processed data to the MES, ignoring the transmission delay of the result delivery; Indicates the time when MES uploads the data after local vehicle processing to DTDC. The time when the data is uploaded to DTDC is ignored. When the vehicle selects the unloading process, the execution delay is as follows: Indicates the time required to upload the original data to MES, Indicates the time required by MES to process this amount of environmental information; the vehicle chooses to execute the solution with the smaller delay, using represents the minimum execution delay of the vehicle, that is: Execution delay on the RSU side It is expressed as follows: Ignore the same Indicates the time required for RSU to upload raw data to MES, Indicates the time required for MES to process this amount of environmental information; S4: With the goal of minimizing the delay in completing the collaborative sensing task, a joint optimization problem of bandwidth allocation, offloading ratio, and computing resource allocation is constructed. In S4, the optimization problem expression for minimizing the processing delay of the collaborative sensing task by combining the collaborative sensing task allocation, offloading, and computing resource allocation problems is: Where B i The bandwidth resources allocated to vehicle i, β i is the unloading decision of vehicle i, μ i is the proportion of computing resources allocated by MES to vehicle i, μ j is the proportion of computing resources allocated by MES to RSUj; S5: We use the multi-agent deep deterministic policy gradient algorithm HAS-MADDPG in a hybrid action space to solve the optimization problem and learn the resource allocation solution with the minimum task completion delay.

2. The digital twin-assisted collaborative perception and edge collaboration resource allocation method according to claim 1 is characterized by: In S1, the IoV scenario consists of a physical IoV system and a digital twin DT space; The physical Internet of Vehicles system includes a central macro base station MBS, several roadside units RSU, edge computing servers MES and networked automatic vehicles CAV; an MES with computing and storage capabilities is deployed next to each RSU and is directly connected to the RSU through a wired link; each CAV and MES is equipped with blockchain technology to assist CAV in selecting the appropriate MES for offloading; CAV is equipped with various sensors and radars to collect data inside and outside the vehicle; when the line of sight is obstructed, CAV obtains the latest global map data from the digital twin DT system to assist driving; MBS plays the role of digital twin data center DTDC in the digital twin-Internet of Vehicles DT-IoV, storing all DT data within its coverage area; in addition to being able to access MBS, the vehicle offloads its own perception data to MES for processing through wireless channels; DT space: each network unit in the physical entity includes CAV, RSU and MES, each network unit sends its current operating status to DT through a real-time channel, and builds VE in the DT space IoV Digital simulation models for different service levels; DT-IoV establishes a three-level simulation platform; first, DT collects data generated during CAV driving, including engine speed, fuel remaining, battery remaining, blockchain data, MES communication and computing resources, blockchain data and communication environment information, to build a single-entity-level simulation model; establish a complex hierarchical model between single entities; in the vehicle-to-infrastructure (V2I) scenario, the DT space stores independent data of CAVs and RSUs, creating a dynamic multi-dimensional, multi-time-scale virtual model of the communication environment between vehicles and roadside units; the DT space uses machine learning algorithms to provide intelligent solutions for task offloading and resource allocation problems in MES scenarios; as physical entities are constructed in the DT space, a DT vehicle network is formed; the DT vehicle network simulates traffic flow control and guidance strategies on the physical vehicle network system and evaluates its effectiveness; Assume that there are N vehicles and M RSUs in the system, a single MES is coupled with one RSU, and the number of MESs is M; RSU, MES, CAV and the communication environment connecting them are all stored in the DT server in MBS; define a matrix of size M×a To cache the MES's Internet of Vehicles digital data DD IoV ; DD of MES at time t IoV Expressed as: Where a represents the number of attributes of each MES; represents the matrix of MES at time t; Define the matrix The first and second columns represent the remaining communication and computing resources of each MES at time t, which are: The residual energy of the MES current is stored in in the other columns.

3. The digital twin-assisted collaborative perception and edge collaboration resource allocation method according to claim 1 is characterized by: In S2, a digital twin DT-assisted edge collaboration method is proposed to provide offloading services for computationally intensive tasks; The dataset storing blockchain information is represented as S j ; Assume that the data set of the blockchain stored in DT is Define the binary number x j To measure the data consistency of candidate MESs, there are: If the blockchain message of candidate MESj is consistent with the block data in DT, then the data consistency is verified, that is, x is established. j =1; otherwise, x j = 0; the better the channel quality between CAV and MESs, the higher the transmission bandwidth obtained by CAV; when sending the same amount of data, CAV consumes less transmission power; the channel coefficient g between CAV and the jth MES is j As the second metric for selecting collaborative MESs; let g th is the threshold of the channel coefficient; Define a binary number y j To measure the channel quality of the j-th candidate communication link between MESj and CAV, we get: When the link channel coefficient between MESj and CAV is equal to or greater than the threshold, the channel quality is guaranteed and y is established. j =1; otherwise, y j = 0; select MESs that guarantee security and high channel quality from MESs, which need to meet x j =1 and y j =1, we get: where q j is a binary number, q j =1 means MESj meets the conditions, q j =0 means that MESj does not meet the conditions.

4. The digital twin-assisted collaborative perception and edge collaboration resource allocation method according to claim 1 is characterized by: In S5, HAS-MADDPG includes the following steps: S61: Initialize the actor network and critic network of each CAV; S62: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each CAV, i.e., the initial state of the environment; S63: At each time slot, based on the current strategy and state, each CAV selects an action and executes it; S64: In each time slot, all CAVs interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool; S65: For each CAV, randomly sample a small batch of samples from the experience pool; S66: For each CAV, calculate the target state value of the critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the actor network; S67: Return to S63 after completing each CAV, otherwise return to S65; S68: Return to S62 after completing each time slot, otherwise return to S63; S69: Stop after the iteration is completed, otherwise return to S62.

Images