Internet of vehicles cooperative sensing resource allocation method based on ISAC

Abstract

The invention relates to an Internet of Vehicles cooperative sensing resource allocation method based on ISAC, and belongs to the technical field of mobile communication. The method comprises the following steps: constructing a multi-layer collaborative perception concerned region scheme based on relative positions; determining a common perceiver according to the perceived concerned area of the current vehicle and the relative positions of the road test unit and other vehicles of the Internet of Vehicles; constructing an ISAC time allocation scheme based on a time division dynamic frame structure, and obtaining total sensing mutual information SMI of radar sensing duration of a common sensor, an average communication rate of communication duration and execution delay of each processing mode; establishing a joint problem of sensing task allocation, unloading and computing resource allocation, which aims at minimizing the completion delay of the collaborative sensing task; and solving an optimization problem by adopting a multi-agent depth deterministic strategy gradient algorithm of a mixed action space to obtain a resource allocation scheme. According to the method, the cooperative sensing resource allocation strategy of the Internet of Vehicles can be optimized, and the execution delay of cooperative sensing is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of mobile communications and relates to an ISAC-based vehicle networking collaborative perception resource allocation method. Background Art

[0002] The perception capability of a single connected automatic vehicle (CAV) has inherent limitations. The emergence of collaborative perception technology based on the Internet of Vehicles can overcome this defect. By fusing the perception data of multiple CAVs and RSUs, the perception range can be greatly expanded and the perception accuracy can be improved. However, there are still some problems with the existing single-layer collaborative perception scheme. The first is that the bandwidth and computing power utilization are saturated or insufficient. In early fusion, excessive raw data transmission may lead to bandwidth saturation and underutilization of computing power. In late fusion, sending object data after local processing may lead to computing power saturation and underutilization of bandwidth. Even in mid-term fusion under dynamic network conditions, the above problems still exist. Secondly, for a single CAV, the road condition information it is most concerned about is not the global map information but the road condition information in a closer range. Generating a complete global environment map requires processing a large amount of perception data, which brings a huge burden to the system's calculation and communication, resulting in high overall latency. Therefore, it is necessary to design a more flexible and efficient collaborative perception scheme to improve system performance.

[0003] Integrated communication and sensing (ISAC) devices play a key role in collaborative perception. Such devices can perform radar perception and communication functions simultaneously, providing the necessary hardware foundation for collaborative perception. By coordinating the use of radar and communication resources, ISAC devices can achieve an effective balance between perception and communication, providing important support for the performance optimization of the entire collaborative perception system. However, existing ISAC devices have some problems in the balance of perception and communication performance and the utilization of local computing resources. Due to the interference between perception and communication functions, it is difficult to achieve the best resource allocation between the two. At the same time, the introduction of on-board computing power is not considered, resulting in high latency in the entire collaborative perception process. By analyzing the impact of different duration allocation ratios on perception and communication performance, and combining on-board computing power, the optimal time allocation ratio range of CAV and RSU is obtained, which can reduce the latency of the entire collaborative perception process. This paper provides a flexible and efficient solution for the application of ISAC devices in collaborative perception scenarios. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a vehicle network collaborative perception resource allocation method based on ISAC, which can optimize the vehicle network collaborative perception resource allocation strategy and reduce the execution delay of collaborative perception.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A method for allocating collaborative sensing resources in an ISAC-based vehicle network includes the following steps:

[0007] S1. Construct a multi-layer collaborative perception area of ​​interest solution based on relative position in the Internet of Vehicles scenario;

[0008] S2, confirming the common perceiver according to the relative position of the perception area of ​​interest of the current vehicle and the road test unit and other connected vehicle vehicles;

[0009] S3, construct the ISAC time allocation scheme based on the time-division dynamic frame structure to obtain the total perception mutual information SMI of the radar sensing duration of the co-perceivers, the average communication rate of the communication duration, and the execution delay of each processing method;

[0010] S4, establish the joint problem of perception task allocation, offloading and computing resource allocation with the goal of minimizing the delay in the completion of collaborative perception tasks;

[0011] S5. Use the multi-agent deep deterministic policy gradient algorithm in mixed action space to solve the optimization problem and obtain the resource allocation plan with the minimum task completion delay.

[0012] Further, in step S1, the Internet of Vehicles scenario includes a number of Internet of Vehicles automated vehicles (CAVs), a number of road test units (RSUs), and a number of mobile edge servers (MESs), the CAVs collect data inside and outside the vehicle through various sensors, a number of RSUs deployed along the road are connected to each other through high-speed wired links, and each RSU is connected to the MES through a wired link;

[0013] The current CAV selects a co-sensor according to the relative positions of the RSU and other CAVs in the sensing area of ​​interest. The processing of the sensing data by the current CAV or the co-sensor includes local processing and offload processing.

[0014] Assume N = {1, ..., i, ..., N} is the set of CAVs in the system, and the perceptual interest area is P PAC , P RSU Indicates the coverage of RSU. The current vehicle v1 cannot perceive P PAC All regions in P PAC Divided into P1, P f and P b , P1 is the area within the visual field of v1, Pf and P b are the occluded areas before and after v1 respectively, then:

[0015] P PAC =P1+P f +P b

[0016] The perception tasks of the three regions are defined as T1, T f , T b , the perception task of CAV perception area of ​​interest is expressed as:

[0017] T PAC =T1+T f +T b

[0018] The ways in which CAVs process perception tasks include local processing or offload processing.

[0019] Further, in step S2, the process of transferring the current vehicle from the coverage of the current RSU to the coverage of the next RSU is divided into the following five stages, and the selection range of the co-perceiver in each stage is:

[0020] In the first stage, T PAC Fully in RSU j The coverage of which the co-perceivers can choose RSU j and in P b , P f CAV;

[0021]

[0022] V f Indicates P PAC CAV before v1, V b Indicates P PAC CAV after v1, V f and V b There may be multiple or none, V V2V is the set of CAVs within the communication range of v1, U j Indicates the RSU of the current range of v1 j , U j+1 Indicates that v1 is about to enter the RSU j+1 ;

[0023] In the second stage, T PAC P f Partial entry into RSU j+1 Coverage of the co-perceiver, the co-perceiver can choose RSU j 、RSU j+1 and in Pb , P f CAV;

[0024]

[0025] In the third stage, T PAC P f Access to RSUs j+1 The coverage area, P b If not entered, its co-perceivers can choose RSU j 、RSU j+1 and in P f CAV;

[0026]

[0027] In the fourth stage, T PAC P b Partial entry into RSU j+1 The coverage of which the co-perceivers can choose RSU j 、RSU j+1 and in P b , P f CAV;

[0028]

[0029] In the fifth stage, T PAC Has fully entered into RSU j+1 Its co-perceivers can choose RSU j+1 and in P b , P f CAV;

[0030]

[0031] The task allocation function is defined as y = TA(x), indicating that T PAC The collaborative task in is assigned to y. According to the relative position information, a group of co-perceivers are selected from RSU and CAV to assist v1 in completing the perception task. Let Ω represent the set of selected co-perceivers, which satisfies:

[0032] Ω=TA(T1)∪TA ρ (T f )∪TA ρ (T b )

[0033] Where TA ρ (T1),TA ρ (T f ),TA ρ (T b) respectively represent the ability to perceive T0, T f 、T b For the perceivers, ρ=1,2,3,4,5 indicates the stage.

[0034] Further, in step S3, an adjustable frame format including S sensing subframes, P computing subframes and Q communication subframes is first constructed, with N subframes in each cycle. s subframes, and the duration of each subframe is τ s , the time allocation decision of each vehicle ISAC device is described as χ vi ={a i ,b i ,c i}, where v i ∈Ω V , a i ,b i ,c i They are v i The corresponding normalized perceived duration, calculated duration, and perceived duration, a i +b i +c i =1, where

[0035] During radar detection, v i During the radar sensing duration a i The average perceptual mutual information (SMI) is expressed as:

[0036]

[0037] Among them, B i Represented as assigned to v i bandwidth; v i The signal and interference plus noise ratio is expressed as:

[0038]

[0039] Where P i Yes i The transmission power, represents the path propagation gain, n r-rad and n r-com Represents other CAV pairs v i Radar interference and communication interference caused by radar signals; N0 represents the thermal noise power spectral density.

[0040] Further, in step S3, during the communication process, v i During the communication duration c i The total communication data volume of the process is:

[0041]

[0042] v i During the communication duration c i The average communication rate is:

[0043]

[0044] in, Indicates v i The signal-to-noise ratio of the communication signal is expressed as:

[0045]

[0046] In the formula, G t and G r represents the antenna transmission gain and antenna receiving gain, represents the communication channel gain from i to j, and They are the radar interference and communication interference respectively.

[0047] Further, in step S3, for each execution delay, the perception task is described as L i ={s i ,d i ,τ i},s i represents the amount of environmental information that needs to be extracted in the perception task, d i Indicates the number of CPU cycles required to process one bit of environmental information, τ i The maximum tolerable delay for the entire perception task is:

[0048] The execution delay of the vehicle's local processing of perception data is:

[0049]

[0050] In the formula, Indicates the CPU cycle frequency assigned to the local task, and defines the maximum CPU cycle frequency of CAV satisfy

[0051] The execution delay of unloading a vehicle to the server is:

[0052]

[0053] In the formula, For RSU j The maximum CPU cycle frequency, μ vi express Assign to v iThe proportion of computing resources.

[0054] The execution delay on the RSU side is:

[0055]

[0056] In the formula, Indicates u j The proportion of computing resources allocated to task processing.

[0057] Further, in step S4, the collaborative sensing task allocation, offloading and computing resource allocation optimization scheme is formulated as the problem of minimizing the collaborative sensing task processing delay. The optimization problem is expressed as:

[0058]

[0059] in For vehicle unloading decisions, Time allocation ratio for vehicle ISAC equipment, is the proportion of computing resources allocated by MES to vehicles, Ω V represents the CAV selected as the co-perceiver, Ω U represents the RSU selected as the co-perceiver;

[0060] C1 represents the selection of common perceivers; C2 represents v i and u j The total processing time cannot exceed the maximum tolerable delay; C3 means v i The sum of the allocated normalized perception, communication and computing time ratios is 1; C4 means that in the same cycle, the amount of communication data cannot exceed the amount of perception data; C5 means that the actual required perception, communication and computing time is not greater than the allocated perception, communication and computing time; C6 means that the sum of the resource ratios allocated to the RSU providing offloading services is not greater than 1.

[0061] Further, in step S5, the optimization problem is transformed into a Markov decision process, and the state space is set as:

[0062]

[0063] Among them, b, g, and l represent the remaining computing resource status of MES, the channel gain of the communication link between CAV and MES, and the connection status between CAV and MES, respectively;

[0064] Action Space:

[0065] Uninstall decision β vi , Time Proportion Allocation The discrete action of and the continuous variable of the computing resource allocation ratio μ are used as the hybrid action of CAV.

[0066] award:

[0067]

[0068] Where β is a constant, when the delay decreases, the reward increases;

[0069] The goal of training the value network is to minimize the prediction error of the state-action value function. Assume that the parameters of each CAV are θ = {θ1, θ2,, ..., θ i ...,θ N}, for each CAV i , use the target network to calculate the target value; calculate the target action a' j , where a' j =μ' j (s') represents the next state s' of the jth agent j The target action under j∈{1,2,...,N} is:

[0070] y i =r i +γQ' i (s',a i )

[0071] Where Q' i For CAV i The target value network;

[0072] Update the value network parameters θ by minimizing the mean squared error loss i :

[0073]

[0074] Where B is a batch drawn from the experience playback area;

[0075] When training the policy network, assume that the policy network parameters of each CAV are ω={ω1,ω2,...ω i ,...ω N}, for each CAV i , use the policy gradient to update the policy network, the policy gradient is:

[0076]

[0077] Update the policy network parameters ω using the gradient ascent method i ;

[0078] For the target network parameters of the value network and the policy network, use soft updates:

[0079] θi '=τθ i '+(1-τ)θ i

[0080] ω i '=τω i '+(1-τ)ω i

[0081] Where τ represents the soft update parameter.

[0082] Further, in step S5, the specific process of using the multi-agent deep deterministic policy gradient algorithm in the hybrid action space to solve the optimization problem includes:

[0083] S51: Initialize the actor network and critic network of each CAV;

[0084] S52: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each CAV, i.e., the initial state of the environment;

[0085] S53: In each time slot, according to the current strategy and state, each CAV selects an action and executes it;

[0086] S54: In each time slot, all CAVs interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool;

[0087] S55: For each CAV, randomly select a small batch of samples from the experience pool;

[0088] S56: For each CAV, calculate the target state value of the critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the actor network;

[0089] S57: After completing each CAV, soft update the target network parameters of the value network and the strategy network, and then return to S53, otherwise return to S55;

[0090] S58: Return to S52 after completing each time slot, otherwise return to S53;

[0091] S59: Stop after the iteration is completed, otherwise return to S52.

[0092] The beneficial effects of the present invention are:

[0093] The present invention proposes a collaborative perception resource allocation scheme for Internet of Vehicles based on ISAC to address the problems of saturated or insufficient bandwidth and computing resource utilization of collaborative perception fusion schemes in existing Internet of Vehicles, large amounts of data for generating global maps, and high system latency caused by the failure of ISAC design to balance perception and communication performance. First, in order to reduce the amount of transmitted data and rationally utilize bandwidth and computing resources, a multi-layer collaborative perception area of ​​interest scheme based on relative position is proposed. Secondly, an ISAC perception, communication, and computing time allocation scheme based on a time-division dynamic frame structure is proposed, the impact of different time allocation ratios on perception and communication mutual information (MI) is analyzed, and time allocation is performed according to the computing power of CAV. The present invention can optimize the collaborative perception resource allocation strategy for Internet of Vehicles and reduce the execution delay of collaborative perception.

[0094] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0096] Figure 1 This is a schematic diagram of the collaborative sensing scenario;

[0097] Figure 2 A schematic diagram of the five stages in the process of a vehicle transferring from the current RSU to the next RSU;

[0098] Figure 3 This is the network structure diagram of the HAS-MADDPG algorithm. DETAILED DESCRIPTION

[0099] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0100] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0101] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0102] See also Figure 1 to Figure 3 , is a method for allocating collaborative perception resources of Internet of Vehicles based on ISAC, which can optimize the collaborative perception resource allocation strategy of Internet of Vehicles and reduce the execution delay of collaborative perception. Specifically, it includes the following steps:

[0103] S1: In the scenario of Internet of Vehicles, a multi-layer collaborative perception area of ​​interest solution based on relative position is proposed, which defines the road condition information in the near range of the car as the perception area of ​​interest to achieve comprehensive perception of the area; Figure 1 Schematic diagram of collaborative sensing scenario.

[0104] The scenario mainly consists of CAV, RSU and MES. CAV is equipped with various sensors to collect data inside and outside the vehicle; several RSUs are deployed at equal distances along the roadside, and the RSUs are connected to each other through high-speed wired links; MES provides computing services for the system, and each RSU is connected to MES through a wired link.

[0105] The multi-layer collaborative sensing area of ​​interest solution based on relative position includes the current CAV selecting a suitable co-perceiver according to the relative positions of the RSU and other CAVs in the sensing area of ​​interest, the current CAV and the co-perceiver perform data fusion, and the current CAV or the co-perceiver can choose to process the sensing data locally or offload the processing. Let N = {1,...,i,...,N} be the set of CAVs in the system.

[0106] The relevant region is described as the Perceptual Area of ​​Interest (PAC) box P PAC , PRSU Represents the coverage of RSU. Due to the occlusion of buildings and other vehicles, the current vehicle v1 cannot perceive P PAC The occluded area of ​​V1 includes the occluded area in front and the blind spot in the back. Therefore, P PAC Divided into P1, P f and P b There are three parts. P1 is the area within the visual field of V1, P f and P b are the occlusions before and after v1, respectively. Therefore,

[0107] P PAC =P1+P f +P b

[0108] When facing occlusion, P cannot be perceived f and P b Vehicles and pedestrians in the area may cause serious traffic accidents. At the same time, for the P1 area, if the computing power of the vehicle is insufficient and it is difficult to process the perception data, it can also request RSU to help through V2I. The perception tasks of the three areas are defined as T1, T f , T b ,Therefore, the perception task of CAV perception area of ​​interest can be expressed as:

[0109] T PAC =T1+T f +T b

[0110] When performing perception task processing, CAVs can choose to process locally or offload processing.

[0111] S2: Analyze the impact of relative position on the selection of co-perceivers. Relative position refers to the relative position of RSU, other CAVs and the perceived area of ​​interest.

[0112] Therefore, it is necessary to select appropriate co-perceivers to complete the P1, P f and P b The perception in the area provides v1 with a good view, and task T1 can be performed by v1. Since the CAV moves very fast and is easily out of the coverage of the current RSU, the focus is on the process of transferring from the coverage of the current RSU to the coverage of the next RSU, such as Figure 2 As shown, the process is divided into five stages:

[0113] In the first stage, T PAC Fully in RSU j The coverage of which the co-perceivers can choose RSU j and in P b, P f CAV;

[0114]

[0115] V f Indicates P PAC CAV before v1, V b Indicates P PAC CAV after v1, V f and V b There may be multiple or none, V V2V is the set of CAVs within the communication range of v1, U j Indicates the RSU of the current range of v1 j , U j+1 Indicates that v1 is about to enter the RSU j+1 ;

[0116] In the second stage, T PAC P f Partial entry into RSU j+1 Coverage of the co-perceiver, the co-perceiver can choose RSU j 、RSU j+1 and in P b , P f CAV;

[0117]

[0118] In the third stage, T PAC P f Access to RSUs j+1 The coverage area, P b If not entered, its co-perceivers can choose RSU j 、RSU j+1 and in P f CAV;

[0119]

[0120] In the fourth stage, T PAC P b Partial entry into RSU j+1 The coverage of which the co-perceivers can choose RSU j 、RSU j+1 and in P b , P f CAV;

[0121]

[0122] In the fifth stage, T PAC Has fully entered into RSUj+1 Its co-perceivers can choose RSU j+1 and in P b , P f CAV.

[0123]

[0124] The task allocation function is defined as y = TA(x), indicating that T PAC The collaborative task in is assigned to y. According to the relative position information, a set of co-perceivers is selected in RSU and CAV to assist v1 in completing the perception task. Let Ω represent the set of selected co-perceivers, which satisfies:

[0125] Ω=TA(T1)∪TA ρ (T f )∪TA ρ (T b )

[0126] Where TA ρ (T1),TA ρ (T f ), TA ρ (T b ) respectively indicate that T0, T f , T b For perceivers, ρ=1,2,3,4,5 indicates the stage they are in.

[0127] S3: Propose a time allocation scheme for ISAC based on time-division dynamic frame structure, analyze the impact of different duration allocation ratios on perception and communication mutual information, so that each vehicle can obtain the best duration allocation ratio; specifically, based on the time-division frame structure and radar communication interference analysis, analyze the impact of different duration allocation ratios on perception MI and communication MI, and consider the computing power of CAV itself, set the allocation of computing time, obtain the best duration allocation ratio for a CAV, and reduce the delay of the collaborative perception process. Including the following steps:

[0128] A S / P / Q adjustable frame format consisting of S sensing subframes, P computing subframes, and Q communication subframes is designed. s subframes, and the duration of each subframe is τ s The time allocation decision of each vehicle ISAC device is described as χ vi ={a i ,b i ,c i}, where v i ∈Ω V , a i ,b i,c i They are v i The corresponding normalized sensing duration, computation duration, and communication duration, a i +b i +c i =1. The calculation duration means that the CAV perception task processing may be selected to be processed locally. If the local processing is b i Indicates the calculation duration. If you choose to offload to MES for processing, then b i is 0.

[0129] Analyze the radar detection process. When v i While performing radar sensing functions, other devices may be performing radar sensing, computing, or communication functions, so v i It may be interfered by radar or communication signals from other devices. Its radar receiving signal can be described as:

[0130] y rad =s rad h rad +n r-com +n r-rad +n

[0131] Where s rad represents the radar transmission signal, h rad represents the propagation gain of the radar response signal in the channel, n r-rad and n r-com Represents other CAV pairs v i The radar interference and communication interference caused by the radar signal, n represents the interference to the signal receiver (including internal thermal noise and other external information interference). Therefore, v i The signal to interference plus noise ratio (SINR) can be expressed as:

[0132]

[0133] Where P i Yes i The transmission power, represents the path propagation gain, N0 represents the thermal noise power spectral density, B i Represented as assigned to v i Bandwidth. During the duration, the radar interference can be expressed as:

[0134]

[0135] In the formula, r j,r Indicates the duration of radar detection Whether j is in the radar sensing time and is subject to communication interference can be expressed as:

[0136]

[0137] Where r j,c Indicates whether j is in the communication time during the radar detection duration. Indicates that the communication duration is from RSU j Transfer to v i The communication signal path loss. t and G r Indicates antenna transmit gain and antenna receive gain.

[0138] v i During the radar sensing duration a i The average sensing mutual information (SMI) can be expressed as:

[0139]

[0140] Analyze the communication process. i The communication duration c i During the process, it will be interfered by radar and communication of other equipment. i The communication duration c i During this period, the communication receiving signal can be described as:

[0141] y com =s com h com +n r-com +n r-rad +n

[0142] Assume v i Communicating with RSUj, the signal and interference plus noise ratio SINR during the communication duration can be expressed as:

[0143]

[0144] represents the communication channel gain from i to j. During the duration, the communication interference can be expressed as:

[0145]

[0146] The radar interference can be expressed as:

[0147]

[0148] Where d i,j Indicates v i The distance between and j.

[0149] vi During the communication duration c i The total communication data volume of the process is:

[0150]

[0151] v i During the communication duration c i The average communication rate is:

[0152]

[0153] Analyzing the process of perceptual data processing, the perceptual task can be described by three terms: i ={s i ,d i ,τ i}. i represents the amount of environmental information that needs to be extracted in the perception task, d i Indicates the number of CPU cycles required to process one bit of environmental information, τ i Represents the maximum tolerable delay for the entire perception task to complete.

[0154] When the vehicle chooses to process perception data locally, the execution delay is as follows:

[0155]

[0156] in Indicates the time required to locally process this amount of environmental information. It represents the time required to upload the processed data to MES for data fusion. Since the scale of the processed results is small, the transmission delay of the results is ignored. When allocating the perception, communication and computing time, the communication allocation time can be considered to be 0, that is, a i +b i =1.

[0157] It represents the perception time of the amount of environmental information that needs to be extracted. The expression is as follows:

[0158]

[0159] definition is the maximum CPU cycle frequency of CAV (i.e., CPU cycles per second), and the CPU frequency allocated to the local task is satisfy In theory, the processing latency for local processing of sensory data tasks is as follows:

[0160]

[0161] in, is the CPU frequency used to process perception tasks on the vehicle.

[0162] In summary, the execution delay of local vehicle processing of perception data is as follows:

[0163]

[0164] When the vehicle offloads to the server to process the perception data, the execution delay is as follows:

[0165]

[0166] in represents the perception time of the amount of environmental information that needs to be extracted, Indicates the time required to upload the original data to MES, Indicates the time required for MES to process this amount of environmental information. It can be calculated by the following formula:

[0167]

[0168] definition For RSU j The processing delay on the MES is expressed as:

[0169]

[0170] where μ vi express Assign to v i The proportion of computing resources.

[0171] In summary, the execution delay of unloading a vehicle to the server is as follows:

[0172]

[0173] When processing data on the RSU side, the execution delay is as follows:

[0174]

[0175] in, represents the perception time of the amount of environmental information that needs to be extracted, It indicates the time required by MES to process this amount of environmental information. The processing delay on MES is expressed as:

[0176]

[0177] in Indicates u j The proportion of computing resources allocated to task processing.

[0178] In summary, the execution delay on the RSU side is:

[0179]

[0180] S4: A resource allocation scheme with the goal of minimizing the delay in completing collaborative sensing tasks, constructing a joint problem of sensing task allocation, offloading, and computing resource allocation;

[0181] In order to provide fast, reliable and efficient Internet of Vehicles services and improve safety and user experience, the time required to complete the task is crucial. This embodiment formulates the collaborative perception task allocation, offloading and computing resource allocation optimization scheme as the problem of minimizing the collaborative perception task processing delay. The optimization problem can be expressed as:

[0182]

[0183] in For vehicle unloading decisions, Time allocation ratio for vehicle ISAC equipment, is the proportion of computing resources allocated by MES to vehicles, Ω V represents the CAV selected as the co-perceiver, Ω U represents the RSU selected as the co-perceiver. The constraints are described in detail below. C1 represents the selection of the co-perceiver, C2 represents v i and u j The total processing time cannot exceed the maximum tolerable delay. C3 means v i The sum of the allocated normalized perception, communication, and computing time ratios is 1. C4 indicates that in the same cycle, the amount of communication data cannot exceed the amount of perception data. C5 indicates that the actual required perception, communication, and computing time is not greater than the allocated perception, communication, and computing time. C6 indicates that the sum of the resource ratios allocated to the RSU providing offloading services is not greater than 1.

[0184] S5: To solve the optimization problem, a multi-agent deep deterministic policy gradient algorithm HAS-MADDPG in a hybrid action space is proposed to learn the resource allocation scheme with the minimum task completion delay; Figure 3 This is the network structure diagram of the HAS-MADDPG algorithm.

[0185] The optimization problem is transformed into a Markov decision process, and the state space is set as:

[0186]

[0187] Among them, b, g, and l represent the remaining computing resource status of MES, the channel gain of the communication link between CAV and MES, and the connection status between CAV and MES, respectively.

[0188] Action Space: Offloading Decisions Time Proportion Allocation The discrete action of and the continuous variable of the computing resource allocation ratio μ are used as the hybrid action of CAV.

[0189] award:

[0190]

[0191] Where β is a constant, which is to increase the reward when the delay decreases.

[0192] The goal of training the value network is to minimize the prediction error of the state-action value function. Assume that the parameters of each CAV are θ = {θ1,θ2,,...,θ i ...,θ N}. For each CAV i , use the target network to calculate the target value. Calculate the target action a' j , where a' j =μ' j (s') represents the next state s' of the jth agent j The target action under is j∈{1,2,...,N}. The target value is:

[0193] y i =r i +γQ' i (s',a i )

[0194] Where Q' i For CAV i target value network.

[0195] Update the value network parameters θ by minimizing the mean squared error loss i :

[0196]

[0197] Where B is a batch drawn from the experience replay area.

[0198] When training the policy network, assume that the policy network parameters of each CAV are ω={ω1,ω2,...ω i ,...ω N}. For each CAV i , use the policy gradient to update the policy network. The policy gradient is:

[0199]

[0200] Update the policy network parameters ω using the gradient ascent method i .

[0201] For the target network parameters of the value network and the policy network, use soft updates:

[0202] θ i '=τθ i '+(1-τ)θ i

[0203] ω i '=τω i '+(1-τ)ω i

[0204] τ represents the soft update parameter.

[0205] The HAS-MADDPG algorithm specifically includes the following steps:

[0206] S51: Initialize the actor network and critic network of each CAV;

[0207] S52: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each CAV, i.e., the initial state of the environment;

[0208] S53: In each time slot, according to the current strategy and state, each CAV selects an action and executes it;

[0209] S54: In each time slot, all CAVs interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool;

[0210] S55: For each CAV, randomly select a small batch of samples from the experience pool;

[0211] S56: For each CAV, calculate the target state value of the critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the actor network;

[0212] S57: After completing each CAV, soft update the target network parameters of the value network and the strategy network, and then return to S53, otherwise return to S55;

[0213] S58: Return to S52 after completing each time slot, otherwise return to S53;

[0214] S59: Stop after the iteration is completed, otherwise return to S52.

[0215] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A method for allocating collaborative perception resources in an ISAC-based Internet of Vehicles, characterized by: It includes the following steps: S1. Construct a multi-layer collaborative perception area of ​​interest solution based on relative position in the Internet of Vehicles scenario; S2, confirming the common perceiver according to the relative position of the perception area of ​​interest of the current vehicle and the road test unit and other connected vehicle vehicles; S3, construct the ISAC time allocation scheme based on the time-division dynamic frame structure to obtain the total perception mutual information SMI of the radar sensing duration of the co-perceivers, the average communication rate of the communication duration, and the execution delay of each processing method; S4, establish the joint problem of perception task allocation, offloading and computing resource allocation with the goal of minimizing the delay in the completion of collaborative perception tasks; S5. Use the multi-agent deep deterministic policy gradient algorithm in mixed action space to solve the optimization problem and obtain the resource allocation plan with the minimum task completion delay.

2. The ISAC-based vehicle networking collaborative perception resource allocation method according to claim 1 is characterized by: In step S1, the Internet of Vehicles scenario includes several Internet of Vehicles automated vehicles (CAVs), several road test units (RSUs), and several mobile edge servers (MESs). The CAVs collect data inside and outside the vehicle through various sensors. Several RSUs deployed along the road are connected to each other through high-speed wired links, and each RSU is connected to the MES through a wired link. The current CAV selects a co-sensor according to the relative positions of the RSU and other CAVs in the sensing area of ​​interest. The processing of the sensing data by the current CAV or the co-sensor includes local processing and offload processing. Assume N = {1, ..., i, ..., N} is the set of CAVs in the system, and the perceptual interest area is P PAC , P RSU Indicates the coverage of RSU. The current vehicle v1 cannot perceive P PAC All regions in P PAC Divided into P1, P f and P b , P1 is the area within the visual field of v1, P f and P b are the occluded areas before and after v1 respectively, then: P PAC =P1+P f +P b The perception tasks of the three regions are defined as T1, T f , T b , the perception task of CAV perception area of ​​interest is expressed as: T PAC =T1+T f +T b The ways in which CAVs process perception tasks include local processing or offload processing.

3. The ISAC-based vehicle networking collaborative perception resource allocation method according to claim 2 is characterized in that: In step S2, the process of transferring the current vehicle from the coverage of the current RSU to the coverage of the next RSU is divided into the following five stages, and the selection range of the co-perceiver in each stage is: In the first stage, T PAC Fully in RSU j The coverage of which the co-perceivers can choose RSU j and in P b , P f CAV; V f Indicates P PAC CAV before v1, V b Indicates P PAC CAV after v1, V f and V b There may be multiple or none, V V2V is the set of CAVs within the communication range of v1, U j Indicates the RSU of the current range of v1 j , U j+1 Indicates that v1 is about to enter the RSU j+1 ; In the second stage, T PAC P f Partial entry into RSU j+1 Coverage, co-perceivers can choose RSU j 、RSU j+1 and in P b , P f CAV; In the third stage, T PAC P f Access to RSUs j+1 The coverage area, P b If not entered, its co-perceivers can choose RSU j 、RSU j+1 and in P f CAV; In the fourth stage, T PAC P b Partial entry into RSU j+1 The coverage of which the co-perceivers can choose RSU j 、RSU j+1 and in P b , P f CAV; In the fifth stage, T PAC Has fully entered into RSU j+1 The co-sensor can select RSUj+1 and b , P f CAV; The task allocation function is defined as y = TA(x), indicating that T PAC The collaborative task in is assigned to y. According to the relative position information, a group of co-perceivers are selected from RSU and CAV to assist v1 in completing the perception task; Let Ω denote the set of selected co-perceivers that satisfies: Ω=TA(T1)∪TA ρ (T f )∪TA ρ (T b ) Where TA ρ (T1),TA ρ (T f ),TA ρ (T b ) respectively represent the ability to perceive T0, T f , T b For the perceivers, ρ=1,2,3,4,5 indicates the stage.

4. The ISAC-based vehicle networking collaborative perception resource allocation method according to claim 3 is characterized by: In step S3, an adjustable frame format including S sensing subframes, P computing subframes and Q communication subframes is first constructed, with N subframes in each cycle. s subframes, and the duration of each subframe is τ s , the time allocation decision of each vehicle ISAC device is described as χ vi ={a i ,b i ,c i }, where v i ∈Ω V , a i ,b i ,c i They are v i The corresponding normalized sensing duration, computation duration, and communication duration, a i +b i +c i =1, where During radar detection, v i During the radar sensing duration a i The average perceptual mutual information SMI is expressed as: Among them, B i Represented as v assigned to i bandwidth; v i The signal and interference plus noise ratio is expressed as: Where P i Yes i The transmission power, represents the path propagation gain, n r-rad and n r-com Represents other CAV pairs v i Radar interference and communication interference caused by radar signals; N0 represents the thermal noise power spectral density.

5. The ISAC-based vehicle networking collaborative perception resource allocation method according to claim 4 is characterized in that: In step S3, during the communication process, v i During the communication duration c i The total communication data volume of the process is: v i During the communication duration c i The average communication rate is: in, Indicates v i The signal-to-noise ratio of the communication signal is expressed as: In the formula, G t and G r represents the antenna transmission gain and antenna receiving gain, represents the communication channel gain from i to j, and They are the radar interference and communication interference respectively.

6. The ISAC-based vehicle networking collaborative perception resource allocation method according to claim 5 is characterized by: In step S3, for each execution delay, the perception task is described as L i ={s i ,d i ,τ i },s i represents the amount of environmental information that needs to be extracted in the perception task, d i Indicates the number of CPU cycles required to process one bit of environmental information, τ i The maximum tolerable delay for the entire perception task is: The execution delay of the vehicle's local processing of perception data is: In the formula, f i v Indicates the CPU cycle frequency assigned to the local task, and defines the maximum CPU cycle frequency F of CAV i v , f i v Satisfy f i v ≤F i v ; The execution delay of unloading a vehicle to the server is: In the formula, For RSU j The maximum CPU cycle frequency, express Assign to v i The proportion of computing resources; The execution delay on the RSU side is: In the formula, Indicates u j The proportion of computing resources allocated to task processing.

7. The ISAC-based vehicle networking collaborative perception resource allocation method according to claim 6 is characterized by: In step S4, the optimization scheme of collaborative sensing task allocation, offloading and computing resource allocation is formulated as the problem of minimizing the collaborative sensing task processing delay. The optimization problem is expressed as: in For vehicle unloading decisions, Time allocation ratio for vehicle ISAC equipment, is the proportion of computing resources allocated by MES to vehicles, Ω V represents the CAV selected as the co-perceiver, Ω U represents the RSU selected as the co-perceiver; C1 represents the selection of common perceivers; C2 represents v i and u j The total processing time cannot exceed the maximum tolerable delay; C3 means v i The sum of the allocated normalized perception, communication and computing time ratios is 1; C4 means that in the same cycle, the amount of communication data cannot exceed the amount of perception data; C5 means that the actual required perception, communication and computing time is not greater than the allocated perception, communication and computing time; C6 means that the sum of the resource ratios allocated to the RSU providing offloading services is not greater than 1.

8. The ISAC-based vehicle networking collaborative perception resource allocation method according to claim 7 is characterized by: In step S5, the optimization problem is transformed into a Markov decision process, and the state space is set as: Where b, g, and l represent the remaining computing resource status of MES, the channel gain of the communication link between CAV and MES, and the connection status between CAV and MES, respectively; Action Space: Uninstall Decision Time Proportion Allocation The discrete action of and the continuous variable of the computing resource allocation ratio μ are used as the hybrid action of CAV. award: Where β is a constant, when the delay decreases, the reward increases; The goal of training the value network is to minimize the prediction error of the state-action value function. Assume that the parameters of each CAV are θ = {θ1, θ2,, ..., θ i ...,θ N }, for each CAV i , use the target network to calculate the target value; calculate the target action a' j , where a' j =μ' j (s') represents the next state s' of the jth agent j The target action under j∈{1,2,...,N} is: y i =r i +γQ’ i (s',a i ) Where Q' i For CAV i The target value network; Update the value network parameters θ by minimizing the mean squared error loss i : Where B is a batch drawn from the experience playback area; When training the policy network, assume that the policy network parameters of each CAV are ω={ω1,ω2,...ω i ,...ω N }, for each CAV i , use the policy gradient to update the policy network, the policy gradient is: Update the policy network parameters ω using the gradient ascent method i ; For the target network parameters of the value network and the policy network, use soft updates: i i '=τθ i '+(1-τ)θ i oh i '=to i '+(1-t)ω i Where τ represents the soft update parameter.

9. The ISAC-based vehicle networking collaborative perception resource allocation method according to claim 8, characterized in that: In step S5, the specific process of solving the optimization problem using the multi-agent deep deterministic policy gradient algorithm in the mixed action space includes: S51: Initialize the actor network and critic network of each CAV; S52: In each iteration, a random process is initialized for action exploration to obtain the initial observation of each CAV, i.e., the initial state of the environment; S53: In each time slot, according to the current strategy and state, each CAV selects an action and executes it; S54: In each time slot, all CAVs interact with the environment to obtain their respective rewards and jump to the next state, storing the experience data in the experience replay pool; S55: For each CAV, randomly select a small batch of samples from the experience pool; S56: For each CAV, calculate the target state value of the critic, calculate the loss function, and minimize the loss to update the critic network, calculate the policy gradient, and update the actor network; S57: After completing each CAV, soft update the target network parameters of the value network and the strategy network, and then return to S53, otherwise return to S55; S58: Return to S52 after completing each time slot, otherwise return to S53; S59: Stop after the iteration is completed, otherwise return to S52.

Images