V2X communication and perception combined resource allocation method

Abstract

The invention relates to a V2X communication and perception combined resource allocation method. According to the method, firstly, a V2X-ISAC network is built in an NR-V2X communication environment, V2I and V2V are combined for communication in the network, and a CRLB boundary is adopted for sensing; and then the network is optimized by using an MADDPG reinforcement learning algorithm, and the communication rate and the sensing precision of the network are maximized at the same time through power control and sub-channel allocation under various service quality constraints. Compared with the prior art, the method has the advantages that the communication rate and the sensing precision are high, the dynamic optimization of resources is realized in a V2X dynamic scene, the balance of the communication rate and the sensing precision is realized, and different application scene requirements are met.

Description

Technical Field

[0001] The present invention relates to the field of vehicle network information transmission technology, and in particular to a V2X communication and perception joint resource allocation method. Background Art

[0002] The development of the 6th Generation Mobile Networks (6G) technology marks the arrival of a highly connected era of wireless communications, and vehicle to everything (V2X) communications, as a key application of 6G, play a key role in creating a safer, more efficient and smarter transportation system. Nowadays, Integrated Sensing and Communication (ISAC) has officially become one of the six major use cases of 6G. As a key application scenario, ISAC will play an important role in the 6G cellular vehicle-to-everything (C-V2X) system based on cellular networks. Therefore, V2X resource optimization is the key to achieving autonomous driving and vehicle connectivity, and plays an important role in improving road safety and realizing intelligent traffic management.

[0003] With the increasing diversity and complexity of wireless networks, traditional V2X resource optimization algorithms such as location-based, clustering-based, and mode selection-based methods face huge challenges. The dynamics and uncertainty of vehicle networks make it difficult to obtain channel state information in real time. In addition, due to the different service requirements of vehicle network users, the proposed optimization problems are often non-convex, which makes traditional algorithms prone to falling into local optimal solutions.

[0004] At the same time, the design goals of traditional wireless communication networks are single, considering only data rate, spectrum efficiency or reliability, which is no longer sufficient to meet the needs of emerging application scenarios.

[0005] In the field of Internet of Vehicles positioning, some studies have begun to use the Cramér-Rao Lower Bound (CRLB) to measure the lower bound of parameter estimation and evaluate the theoretical limit of positioning accuracy. However, it is still impossible to effectively realize the dynamic allocation of resources in dynamic scenarios, and the communication rate and perception accuracy need to be improved.

[0006] For example, in the invention patent with publication number CN116346860A, a method for enhancing information transmission of synaesthesia-fusion vehicle network based on deep reinforcement learning is disclosed. This method designs the Cramer-Rao lower bound of the target vehicle relative to the roadside unit as a perception constraint; obtains the ISAC signal and rate between all target vehicles and the roadside unit; based on the ISAC signal and rate and perception constraints, establishes a maximum sum rate model related to the beamforming and transmission power of the roadside unit; solves the maximum sum rate model based on the deep reinforcement learning method, and obtains the optimal beamforming and transmission power by jointly optimizing the beamforming and transmission power of the roadside unit. However, the constraints used are too single and cannot effectively maximize the communication rate and perception accuracy of the network at the same time. Summary of the invention

[0007] The purpose of the present invention is to provide a V2X communication and perception joint resource allocation method in order to overcome the defects of the above-mentioned prior art.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] According to one aspect of the present invention, a V2X communication and perception joint resource allocation method is provided, and the method specifically comprises the following steps:

[0010] S1. Build a V2X-ISAC network, which uses joint communication between vehicles and base stations and between vehicles, namely V2I and V2V joint communication, and vehicles in the network travel at a preset speed;

[0011] S2, modeling the communication links of V2I and V2V, calculating the signal-to-interference-to-noise ratio of the V2I link and the V2V link and determining the communication rate of the V2I link while considering the interference between the links;

[0012] S3. In the NR-V2X communication environment, the V2V communication link performs side link positioning, transmits data packets between vehicles, and simultaneously performs target detection and positioning on surrounding passive objects to obtain the perceived CRLB boundary;

[0013] S4. Optimize the V2X-ISAC network, establish a deep reinforcement learning model, define the state, action space and reward function, and use the MADDPG algorithm to solve the optimization problem. Under multiple service quality constraints, by maximizing the communication rate of the V2I link and the perception accuracy of CRLB, the optimal allocation plan of communication resources and perception resources is output.

[0014] As a preferred technical solution, the V2I communication link described in S2 is modeled as follows: The V2V communication link is modeled as In the V2X-ISAC network, resources are scheduled in sub-channel units and evenly allocated to the V2I link first, and the V2I uplink spectrum sub-channel resources are multiplexed by multiple V2V links;

[0015] The sub-channel allocation of the V2I link is modeled as Indicates that the nth subchannel is assigned to the lth V2I link, otherwise

[0016] The sub-channel allocation of the V2I link is modeled as Represents the reuse of the V2V link, indicates that the nth subchannel is assigned to the kth V2V link, otherwise

[0017] As a preferred technical solution, the specific calculation formula for the signal to noise ratio of the V2I link and the V2V link in S2 is:

[0018]

[0019]

[0020] In the formula, is the channel power gain from the transmitter of the lth V2I link to the nth subchannel; Large-scale fading effects, including path loss and shadowing; is the small-scale fading component; is the received signal-to-noise ratio of the lth V2I link on the nth subchannel; is the transmit power of the lth V2I link; is the interference from other V2I links; represents additive white Gaussian noise, which follows a normal distribution Indicates the allocation of the nth subchannel; is the transmission power of the kth V2V link; is the channel power gain from the transmitter of the kth V2V link to the nth subchannel; is the received signal-to-noise ratio of the kth V2V link on the nth subchannel; is the interference from other V2V links; is the binary spectrum allocation factor, representing the sub-channel allocation of the V2I link; is the interference of the lth V2I link to the kth V2V link; is the interference of the k′th V2V link transmitter to the kth V2V link receiver.

[0021] As a preferred technical solution, cyclic prefix OFDM signals are used for communication and perception in the NR-V2X communication environment in S3.

[0022] As a preferred technical solution, the state input of the reinforcement learning model in S4 is defined as the channel gain of all V2I links, the channel gain of the V2V link and the interference between the links, and its state input expression is:

[0023] s t ={g(t),I l (t),I k (t)},

[0024]

[0025] Where g(t) represents the channel gain and interference channel of all V2I links and V2V links, I l (t), represents the interference power of the first V 2I link, I k (t) represents the interference power of the kth V 2V link, represents the kth V2V channel gain, represents the interference channel from the kth V2V transmitter to the kth V2V receiver on the nth subchannel, represents the interference channel from the lth V2I transmitter to the kth V2V receiver on the nth subchannel, represents the interference channel from the kth V2V transmitter to the BS on the lth subchannel, represents the channel power gain from the transmitter of the lth V2I link to the nth subchannel.

[0026] As a preferred technical solution, the action space of the reinforcement learning model in S4 is defined as the sub-channel allocation and power allocation of all V2V links, and its specific expression is:

[0027] a t ={a1(t),a2(t)},

[0028]

[0029] a1(t) represents the allocation of subchannels; Indicates whether the kth V2V link reuses the lth subchannel, where indicates that the lth subchannel is assigned to the kth V2V link, otherwise a2(t) represents the power distribution, represents the power of the kth V2V link on the lth subchannel.

[0030] As a preferred technical solution, the reward of the reinforcement learning model in S4 is defined as the total communication rate of the V2I link and the perception accuracy of the V2V positioning;

[0031] The above mentioned reward also takes into account the minimum SINR constraints of the V2I link and the V2V link, and the specific expression of the constraint is:

[0032]

[0033] In the formula, Constrain its V2I communication rate to be maximized; Constrain its CRLB limit to be minimized, that is, the perceived V2V positioning accuracy is maximized; and The constraint is to maximize the SINR constraints on V2I and V2V; L and K represent the number of V2I links and V2V links; a(t) and b(t) represent the satisfaction of the QoS constraints of all V2I links and V2V links, where a l (t) indicates whether the lth V2I link meets the minimum SINR requirement. a l (t)=1, otherwise, a l (t) = 0; similarly b k (t) indicates whether the kth V2V link can meet the minimum SINR requirement. b k (t) = 1, otherwise b k (t)=0.

[0034] As a preferred technical solution, the perceived CRLB limit is obtained in S3, and its calculation formula is:

[0035]

[0036] where u s (t) is the CRLB of the positioning estimation error, SNR is the signal-to-interference-noise ratio perceived by the vehicle, and P T (t) is the transmit power of the V2V link, f c (t) is the carrier frequency, G is the antenna gain, σ is the radar cross section RCS, P n is the noise power, k B is the Boltzmann constant, T0 is the reference temperature, F is the receiver noise, and d is the distance between the target vehicle detected by the sensor and the transmitting vehicle.

[0037] As a preferred technical solution, the expression of the overall optimization objective of optimizing the network using the MADDPG reinforcement learning algorithm in S4 is:

[0038]

[0039] Where u(t) is the overall index of synaesthesia integration, u c (t) is the communication utility index, α u , β u is the proportionality coefficient, u s (t) is the perceived utility index.

[0040] As a preferred technical solution, the multiple service quality constraints in S4 include constraints C1 to C9, where constraint C1 means that each V2I link can only use one subchannel, and each subchannel can be occupied by at most one V2I link; constraint C2 means that each V2V link can reuse the subchannel resources of at most one V2I link, but each subchannel can be reused by multiple V2V links; constraints C3 and C4 mean that the power of the V2I link and the V2V link cannot exceed the maximum power limit and Constraint C5 indicates that the power of the V2I link and the V2V link cannot be less than 0; Constraints C6 and C7 indicate that the SINR of the V2I link and the V2V link cannot be lower than the minimum SINR threshold. and Constraint C8 represents the Cramer-Rao lower bound for V2V Sidelink positioning estimation The minimum positioning accuracy requirement τ cannot be exceeded; constraint C9 indicates that the entire network resources are divided into two parts: communication resources and perception resources; the expressions of each constraint are as follows:

[0041]

[0042] C9:N c +N s ≤N total

[0043] In the formula, is the binary spectrum allocation factor, representing the sub-channel allocation of the V2I link; L is the number of V2I links indicates that the nth subchannel is assigned to the kth V2V link, otherwise K is the number of V2V links is the transmission power of the lth V2I link; is the transmission power of the kth V2V link; is the received signal-to-interference-noise ratio of the lth V2V link; is the received signal-to-interference-noise ratio of the kth V2I link; and is the minimum threshold of the received signal-to-interference-noise ratio; is the Cramer-Rao lower bound of the distance d estimation error; τ is the minimum positioning accuracy requirement; Ntotal is the overall resource; N c N is the communication resource; s For perception resources.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] 1. In the present invention, firstly, a V2X-ISAC network is built in the NR-V2X communication environment, in which V2I and V2V are jointly communicated, and CRLB boundaries are used for perception; then the MADDPG reinforcement learning algorithm is used to optimize the network, and under multiple service quality constraints, power control and sub-channel allocation are used to maximize the communication rate and positioning perception accuracy of the network, so that it can realize dynamic optimization of resources in V2X dynamic scenarios, and effectively adapt to the requirements of different application scenarios.

[0046] 2. In the resource allocation scheme of the present invention, resources are scheduled in sub-channel units and are first evenly allocated to the V2I link, and the V2I uplink spectrum sub-channel resources are multiplexed by multiple V2V links; its resources are divided into communication resources and perception resources. For scenarios with high communication requirements or high positioning requirements, they can be optimized by adjusting the resource allocation weights to achieve a wide range of applicable scenarios.

[0047] 3. In a V2X communication and perception joint resource allocation method of the present invention, a V2X-ISAC network is constructed. The network adopts joint communication between vehicles and base stations and between vehicles, that is, V2I and V2V joint communication. The V2I link meets the demand for high communication rate, and the V2V link meets the demand for high-precision positioning, thereby achieving the effect of simultaneously meeting high communication requirements and high positioning requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of steps of a V2X communication and perception joint resource allocation method in the present invention;

[0049] Figure 2 It is a schematic diagram of the V2X ISAC network scenario in the present invention;

[0050] Figure 3 It is a structural diagram of a unit in the MADDPG algorithm structure of the present invention;

[0051] Figure 4 It is a flow chart of the MADDPG algorithm in the present invention;

[0052] Figure 5 is the convergence diagram of the MADDPG algorithm in the embodiment;

[0053] Figure 6It is a reward graph of the MADDPG algorithm compared with other algorithms in the embodiment;

[0054] Figure 7 is a V2I rate graph of the MADDPG algorithm compared with other algorithms in the embodiment;

[0055] Figure 8 is a CRLB graph of the MADDPG algorithm under different resource allocation weights in the embodiment;

[0056] Fig. 9 It is a diagram showing the trade-off relationship between communication and perception of the MADDPG algorithm in the embodiment compared with other algorithms. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0058] The development of 6G technology marks the arrival of a highly connected era of wireless communications, bringing users an unprecedented hyper-connected experience. In this context, V2X communication, as a key application of 6G, plays a key role in creating a safer, more efficient and smarter transportation system. As a key application scenario, ISAC will play an important role in the 6G C-V2X communication system. Therefore, V2X resource optimization is the key to achieving autonomous driving and vehicle interconnection, and plays an important role in improving road safety and realizing intelligent traffic management.

[0059] With the increasing diversity and complexity of wireless networks, traditional V2X resource optimization algorithms such as location-based, clustering-based, and mode selection-based methods face huge challenges. The dynamic and uncertain nature of vehicle networks makes it difficult to obtain channel state information in real time. In addition, due to the different service requirements of vehicle network users, the proposed optimization problems are often non-convex, which makes traditional algorithms prone to falling into local optimal solutions. In contrast, Deep Reinforcement Learning (DRL) has proven to have a powerful ability to solve complex problems. DRL can make the best decisions through interactive learning with the environment to maximize long-term cumulative rewards, thereby achieving adaptive dynamic resource allocation.

[0060] At the same time, the design goals of traditional wireless communication systems, such as only considering data rate, spectrum efficiency and reliability, are no longer sufficient to meet the needs of emerging application scenarios. The integration of sensing and communication can improve spectrum and energy efficiency, enable vehicles to better perceive the surrounding environment and make smarter decisions. The integration of sensing and communication in 6G vehicle networking is expected to meet the needs of ultra-low latency, ultra-high reliability, ultra-high data rate, and high-precision sensing in intelligent transportation such as autonomous driving and remote driving. At present, most studies use sensor resolution, root mean square error, and positioning accuracy as sensor performance indicators. In the field of positioning, most use CRLB to measure the lower bound of parameter estimation and evaluate the theoretical limit of positioning accuracy, that is, the minimum possible value of positioning error.

[0061] Therefore, in order to cope with dynamically changing scenarios and meet the needs of some emerging applications such as autonomous driving and remote driving in intelligent transportation, which require both high communication rate and high-precision positioning perception, this solution proposes a V2X communication and perception joint resource allocation method based on MADDPG. In a single base station scenario with multiple V2I and V2V links, the V2I link is mainly used to meet the high data rate requirements, while the V2V link is mainly used to transmit safety information and perform side link positioning to ensure the required positioning accuracy. CRLB is used as a measurement indicator for positioning perception, and a deep reinforcement learning MADDPG algorithm is used to solve the resource optimization problem to achieve dynamic resource allocation in dynamic scenarios. This method can achieve high communication rate and perception accuracy, and at the same time derive a trade-off relationship between communication and perception.

[0062] Example

[0063] In this embodiment, a V2X communication and perception joint resource allocation method is applied. The method first builds a V2X-ISAC network in the NR-V2X communication environment. In the network, V2I and V2V are used for joint communication, and CRLB boundaries are used for perception. Then, the MADDPG reinforcement learning algorithm is used to optimize the network. Under multiple service quality constraints, power control and sub-channel allocation are used to maximize the communication rate and perception accuracy of the network. The method flow is as follows: Figure 1 As shown, the specific steps include:

[0064] S1. Build a V2X-ISAC network, which uses joint communication between vehicles and base stations and between vehicles, namely V2I and V2V joint communication, and vehicles in the network travel at a preset speed;

[0065] S2, modeling the communication links of V2I and V2V, calculating the signal-to-interference-to-noise ratio of the V2I link and the V2V link and determining the communication rate of the V2I link while considering the interference between the links;

[0066] S3. In the NR-V2X communication environment, the V2V communication link performs side link positioning, transmits data packets between vehicles, and simultaneously performs target detection and positioning on surrounding passive objects to obtain the perceived CRLB boundary;

[0067] S4. Optimize the V2X-ISAC network, establish a deep reinforcement learning model, define the state, action space and reward function, and use the MADDPG algorithm to solve the optimization problem. Under multiple service quality constraints, by maximizing the communication rate of the V2I link and the perception accuracy of CRLB, the optimal allocation plan of communication resources and perception resources is output.

[0068] In this embodiment, the overall optimization goal of optimizing the network using the MADDPG reinforcement learning algorithm is to maximize the communication rate and perception accuracy of the network through power control and subchannel allocation under multiple service quality constraints. Its expression is:

[0069]

[0070] Where u(t) is the overall index of synaesthesia integration, u c (t) is the communication utility index, α u For u ,u s (t) is the perceived utility index

[0071] Among them, the various service quality constraints include constraints C1 to C9, where constraint C1 means that each V2I link can only use one subchannel, and each subchannel can be occupied by at most one V2I link; constraint C2 means that each V2V link can reuse the subchannel resources of at most one V2I link, but each subchannel can be reused by multiple V2V links; constraints C3 and C4 mean that the power of the V2I link and the V2V link cannot exceed the maximum power limit and Constraint C5 indicates that the power of the V2I link and the V2V link cannot be less than 0; Constraints C6 and C7 indicate that the SINR of the V2I link and the V2V link cannot be lower than the minimum SINR threshold. and Constraint C8 represents the Cramer-Rao lower bound for V2V Sidelink positioning estimation The minimum positioning accuracy requirement τ cannot be exceeded; constraint C9 indicates that the entire network resources are divided into two parts: communication resources and perception resources; the expressions of each constraint are as follows:

[0072]

[0073] C9:N c +N s ≤N total

[0074] In the formula, is the binary spectrum allocation factor, representing the sub-channel allocation of the V2I link; L is the number of V2I links indicates that the nth subchannel is assigned to the kth V2V link, otherwise K is the number of V2V links is the transmission power of the lth V2I link; is the transmission power of the kth V2V link; is the received signal-to-interference-noise ratio of the lth V2V link; is the received signal-to-interference-noise ratio of the kth V2I link; and is the minimum threshold of the received signal-to-interference-noise ratio; is the Cramer-Rao lower bound of the distance d estimation error; τ is the minimum positioning accuracy requirement; N total is the overall resource; N c N is the communication resource; s For perception resources.

[0075] In this embodiment, a network of V2X communication and perception joint resource allocation method based on MADDPG is created, such as Figure 2 As shown in Figure 1, in a single-base station V2X ISAC network, there are L V2I links and K V2V links. The V2I link is mainly used to meet the high communication rate requirements, while the V2V link is mainly used to meet the high-precision positioning requirements. The collections of V2I links and V2V links are defined as and In the network, resources are divided into communication resources and perception resources. Resources are scheduled in sub-channel units. Resources are first evenly allocated to the V2I link. In order to improve spectrum utilization, the V2I uplink spectrum sub-channel resources can be multiplexed by multiple V2V links. Indicates that the nth subchannel is assigned to the lth V2I link, otherwise Its expression is:

[0076]

[0077] In the formula, is a binary spectrum allocation factor, representing the sub-channel allocation of the V2I link.

[0078] same, represents the multiplexing of the kth V2V link in the nth sub-channel, and its expression is:

[0079]

[0080] In the formula, indicates that the nth subchannel is assigned to the kth V2V link, otherwise N c The total number of allocated communication sub-channels.

[0081] In this embodiment, the V2I communication link and the V2V communication link are modeled, and the channel power gain from the transmitter of the lth V2I link to the nth subchannel is defined as Its expression is:

[0082]

[0083] In the formula, is the channel power gain, represents the large-scale fading effects, including path loss and shadowing effects, It is a small-scale fading component.

[0084] Therefore, the received signal-to-noise ratio (SINR) of the lth V2I link on the nth subchannel can be expressed as follows:

[0085]

[0086] In the formula, is the received signal-to-noise ratio of the lth V2I link, represents additive white Gaussian noise, which follows a normal distribution is the transmission power of the lth V2I link, is the transmission power of the kth V2V link. is the interference from other V2V links, is the interference channel from the kth V2V transmitter to the BS on the nth subchannel, is the interference channel from the i-th V2V transmitter to the BS on the n-th subchannel.

[0087] Similarly, on the nth subchannel, the received signal-to-noise ratio (SINR) result of the kth V2I link is expressed as follows:

[0088]

[0089] In the formula, is the received signal-to-noise ratio of the kth V2I link, is the channel gain of the kth V2V link, is the interference of the lth V2I link to the kth V2V link, For the kth ′ The interference of the kth V2V link transmitter to the kth V2V link receiver, is the interference of the lth V2V link transmitter to the kth V2V link receiver, is the transmission power of the lth V2I link, is the transmission power of the kth V2V link, is the interference from other V2V links, is additive white Gaussian noise, Represents the multiplexing status of the V2V link.

[0090] Therefore, W c is the communication bandwidth allocated to each V2I link, and the capacity of the lth V2I link is expressed as:

[0091]

[0092] In the formula, is the capacity of the lth V2I link, is the binary spectrum allocation factor, representing the sub-channel allocation of the V2I link, is the received signal-to-noise ratio of the kth V2I link.

[0093]

[0094] In the formula, u c (t) is the communication utility indicator, l represents the lth V2I link, and n represents the nth subchannel.

[0095] In this embodiment, in the NR-V2X communication environment, NR-V2X uses a cyclic prefix OFDM signal for transmission and side link positioning, where the complex baseband signal of the mth symbol is expressed as:

[0096]

[0097] In the formula, s m (t) is the complex baseband signal of the mth symbol, P n,m is the transmission power on the nth subcarrier, x n,m represents the complex data on the mth subcarrier, Δf is the subcarrier spacing, T sym is the duration of an OFDM symbol.

[0098] In this embodiment, the signal transmitted by OFDM, reflected by the object and captured by the i-th vehicle can be expressed as follows:

[0099]

[0100] In the formula, y m,nis the signal captured by the i-th vehicle, α is the channel coefficient, m is the m-th OFDM symbol, and τ is the delay of the signal reflected by the target.

[0101] Therefore, the CRLB of the positioning estimation error for distance d can be written as follows:

[0102]

[0103] In the formula, u s (t) is the perceived utility index, is the Cramer-Rao lower bound of the distance d estimation error, N is the number of subcarriers, c is the speed of light, SNR is the signal-to-noise ratio perceived by the vehicle, and P R (t) is the perceived received power, P n is the noise power, k B is the Boltzmann constant, T0 is the reference temperature, F is the receiver noise, W s For P T (t) is the transmission power of the V2V link, G is the antenna gain, σ is the radar cross section RCS, f c (t) is the carrier frequency, and d is the distance between the target vehicle detected by the perception and the transmitting vehicle.

[0104] In this embodiment, the structure of a unit in the MADDPG algorithm structure is shown as follows: Figure 3 As shown in the figure, the MADDPG algorithm structure is composed of three cascaded unit structures. The MADDPG algorithm allows multiple agents to cooperate or compete effectively through centralized training and distributed execution. In the MADDPG algorithm structure, each V2V link is used as an agent, and each agent has its own observation space and action space. The algorithm first initializes the environment and agents, and each agent has its own participant (Acto r ) network and critic network, the Actor network generates actions, and the Critic network is used to evaluate the joint state and operations of all agents.

[0105] In this embodiment, the agent stores the observed state, the action performed, the reward received, and the next state in the experience pool for subsequent learning and updating. After a fixed time step, each V2V link agent samples a batch of memories from the experience pool and uses these memories to update its own Actor and Criti c This process involves calculating the loss and performing gradient descent to optimize the agent’s policy and value function.

[0106] In this embodiment, the process of the V2X communication and perception joint resource allocation method based on MADDPG is as follows: Figure 4 As shown, the specific process is as follows:

[0107] First, a vehicle environment network is generated through simulation, including multiple V2I links and V2V links, where each V2V link is used as an agent for training and learning and generating corresponding decisions. Then, the critic network and actor network, as well as the corresponding network parameters, are initialized for each agent. At the same time, a playback buffer (experience pool) and a noise mechanism for exploration are set for each V2V link agent. Each stage (Episode) starts from the initial state of the environment. In each time step (Time Step), each V2V link agent performs operations including: selecting actions, executing actions, and storing transformations.

[0108] The specific steps of selecting an action are: selecting an action by adding exploration noise based on the output of its participant network, where the action is the power selection and sub-channel selection of the V2V link.

[0109] The specific steps of executing the action are: execute the selected action and observe the reward and new state given by the environment. The reward takes into account the utility of communication and perception in the system, including the communication rate of V2I and the perception accuracy of V2V, as well as the QoS requirements of V2V link and V2I link.

[0110] The specific steps of storage conversion are: storing the observed rewards and new states in the replay buffer for subsequent learning.

[0111] In this embodiment, the state input of the reinforcement learning model is the channel gain of all V2I links, the channel gain of the V2V link and the interference between the links, and its state input expression is:

[0112] s t ={g(t),I l (t),I k (t)},

[0113]

[0114] Where g(t) represents the channel gain and interference channel of all V2I links and V2V links, I l (t), represents the interference power of the first V 2I link, I k (t) represents the interference power of the kth V 2V link, represents the kth V2V channel gain, represents the interference channel from the kth V2V transmitter to the kth V2V receiver on the nth subchannel, represents the interference channel from the lth V2I transmitter to the kth V2V receiver on the nth subchannel, represents the interference channel from the kth V2V transmitter to the BS on the lth subchannel, represents the channel power gain from the transmitter of the lth V2I link to the nth subchannel.

[0115] In this embodiment, the action space of the reinforcement learning model is defined as the sub-channel allocation and power allocation of all V2V links, and its specific expression is:

[0116] a t ={a1(t),α2(t)},

[0117]

[0118] a1(t) represents the allocation of subchannels; Indicates whether the kth V2V link reuses the lth subchannel, where indicates that the lth subchannel is assigned to the kth V2V link, otherwise a2(t) represents the power distribution, represents the power of the kth V2V link on the lth subchannel.

[0119] In this embodiment, the reward of the reinforcement learning model is defined as the total communication rate of the V2I link and the perception accuracy of the V2V positioning;

[0120] The reward also takes into account the minimum SINR constraints of the V2I link and the V2V link. The specific expression of the constraint is:

[0121]

[0122] In the formula, Constrain its V2I communication rate to be maximized; Constrain its CRLB limit to be minimized, that is, the perceived V2V positioning accuracy is maximized; and The constraint is to maximize the SINR constraints on V2I and V2V; L and K represent the number of V2I links and V2V links; a(t) and b(t) represent the satisfaction of the QoS constraints of all V2I links and V2V links, where a 1 (t) indicates whether the lth V2I link meets the minimum SINR requirement. a 1 (t)=1, otherwise, a 1 (t) = 0; similarly b k (t) indicates whether the kth V2V link can meet the minimum SINR requirement. b k (t) = 1, otherwise b k(t)=0.

[0123] In this embodiment, after each time step, the state of each V2V link agent is updated to prepare for the next time step. After all time steps of each Episode are completed, a batch of random samples are sampled from the playback buffer for conversion. The critic network of each agent is updated by minimizing the following loss, and the actor network is updated using the sampled policy gradient. Finally, the reward converges, and the system learns the optimal strategy in this vehicle networking scenario, achieving the application requirements of both communication and perception.

[0124] In this embodiment, the convergence diagram of the MADDPG algorithm is as follows: Figure 5 As shown, it can be seen that as the number of iterations progresses, the loss initially rises before starting to decrease and eventually reaches a stable point. This pattern is attributed to the dynamic nature of the environment, which means that the training process does not stabilize immediately. Instead, it converges after a period of continuous interactive learning.

[0125] In this embodiment, the reward diagram of the MADDPG algorithm compared with other algorithms is as follows: Figure 6 As shown in the figure, the circle marks the case of using the MADDPG algorithm; the square marks the case of using the DDPG algorithm; the triangle marks the case of using the PPO algorithm; and the diamond marks the case of using the Random algorithm. Figure 6 It can be clearly seen that the MADDPG algorithm proposed in this scheme obtains the highest reward. Compared with the DDPG algorithm, the MADDPG algorithm enables multiple agents to collaborate or compete in the environment, and the behaviors of the agents depend on each other. Each agent has its own actor network but shares a critic network. This architecture allows each agent to make decisions based on local observations, while the critic network uses global information to evaluate the behavior of all agents, so it can manage more complex and dynamic vehicle scenarios.

[0126] In this embodiment, the V2I rate diagram of the MADDPG algorithm compared with other algorithms is as follows: Figure 7 As shown in the figure, the circle marks the case of using the MADDPG algorithm; the square marks the case of using the DDPG algorithm; the triangle marks the case of using the PPO algorithm; and the diamond marks the case of using the Random algorithm. Figure 7It can be clearly seen that the V2I rates of several algorithms show a gradual upward trend as the number of iterations increases. The MADDPG algorithm in this scheme performs well in dynamic scenarios and achieves the highest rate, which is attributed to its multi-agent interaction with the environment. At the same time, although the dynamic characteristics of vehicle movement cause the V2I rate to fluctuate, the MADDPG algorithm in this scheme shows a stable trend over time. The DDPG and PPO algorithms experience oscillations, indicating that their learning ability is weak in the face of increasing complexity and dynamics.

[0127] The specific method of using the CRLB limit for perception is to calculate the positioning estimation error CRLB, and its formula is:

[0128]

[0129] where u s (t) is the CRLB of the positioning estimation error, SNR is the signal-to-interference-noise ratio perceived by the vehicle, and P T (t) is the transmit power of the V2V link, f c (t) is the carrier frequency, G is the antenna gain, σ is the radar cross section RCS, P n is the noise power, k B is the Boltzmann constant, T0 is the reference temperature, F is the receiver noise, and d is the distance between the target vehicle detected by the sensor and the transmitting vehicle.

[0130] In this embodiment, the CRLB diagram of the MADDPG algorithm under different resource allocation weights is as follows: Figure 8 As shown in the figure, the circle marks the situation when the weight w1 is 0.3 when the MADDPG algorithm is used; the square marks the situation when the weight w1 is 0.5 when the MADDPG algorithm is used; the triangle marks the situation when the weight w1 is 0.7 when the MADDPG algorithm is used. It can be clearly seen that, assuming that the total system bandwidth remains unchanged, the system resources are divided into communication and perception resources. As the communication resource weight w1 increases, the V2I rate will gradually increase, but the perception accuracy CRLB will gradually decrease, and the CRLB value will gradually increase. Therefore, in scenarios with different application requirements, the method in this scheme can dynamically adjust the weights allocated to communication and positioning perception resources to optimize the allocation results.

[0131] In this embodiment, the trade-off relationship between communication and perception of the MADDPG algorithm compared with other algorithms is shown in the figure below: Fig. 9 As shown in the figure, the circle mark indicates the case of using the MADDPG algorithm; the square mark indicates the case of using the DDPG algorithm; the triangle mark indicates the case of using the PPO algorithm; and the diamond mark indicates the case of using the Random algorithm. Fig. 9The trade-off between V2X ISAC communication rate and perception accuracy in this scenario is demonstrated. The proposed MADDPG algorithm achieves the highest rate and positioning accuracy compared to the DDPG and PPO algorithms. When the perception accuracy increases, that is, CRLB decreases, the rate decreases. Similarly, when the rate increases, the perception accuracy decreases. When w1 is 0.5, both CRLB and V2I communication rate are relatively good results.

[0132] In summary, a V2X communication and perception joint resource allocation method in this scheme maximizes the communication rate and positioning perception accuracy of the network through power control and sub-channel allocation under multiple service quality constraints, so that it can realize dynamic optimization of resources in V2X dynamic scenarios and effectively adapt to the requirements of different application scenarios.

[0133] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A V2X communication and perception joint resource allocation method, characterized in that: The method specifically comprises the following steps: S1. Build a V2X-ISAC network, which uses joint communication between vehicles and base stations and between vehicles, namely V2I and V2V joint communication, and vehicles in the network travel at a preset speed; S2, modeling the communication links of V2I and V2V, calculating the signal-to-interference-to-noise ratio of the V2I link and the V2V link and determining the communication rate of the V2I link while considering the interference between the links; S3. In the NR-V2X communication environment, the V2V communication link performs side link positioning, transmits data packets between vehicles, and simultaneously performs target detection and positioning on surrounding passive objects to obtain the perceived CRLB boundary; S4. Optimize the V2X-ISAC network, establish a deep reinforcement learning model, define the state, action space and reward function, and use the MADDPG algorithm to solve the optimization problem. Under multiple service quality constraints, by maximizing the communication rate of the V2I link and the perception accuracy of CRLB, the optimal allocation plan of communication resources and perception resources is output.

2. A V2X communication and perception joint resource allocation method according to claim 1, characterized in that: The V2 I communication link described in S2 is modeled as The V2V communication link is modeled as In the V2X-ISAC network, resources are scheduled in sub-channel units and evenly allocated to the V2I link first, and the V2I uplink spectrum sub-channel resources are multiplexed by multiple V2V links; The sub-channel allocation of the V2I link is modeled as Indicates that the nth subchannel is assigned to the lth V2I link, otherwise The sub-channel allocation of the V2I link is modeled as Represents the reuse of the V2V link, indicates that the nth subchannel is assigned to the kth V2V link, otherwise 3. A V2X communication and perception joint resource allocation method according to claim 1, characterized in that: The specific calculation formula of the signal to interference noise ratio of the V2I link and the V2V link in S2 is: In the formula, is the channel power gain from the transmitter of the lth V2I link to the nth subchannel; Large-scale fading effects, including path loss and shadowing; is the small-scale fading component; is the received signal-to-noise ratio of the lth V2I link on the nth subchannel; is the transmit power of the lth V2I link; is the interference from other V2I links; represents additive white Gaussian noise, which follows a normal distribution Indicates the allocation of the nth subchannel; is the transmission power of the kth V2V link; is the channel power gain from the transmitter of the kth V2V link to the nth subchannel; is the received signal-to-noise ratio of the kth V2V link on the nth subchannel; is the interference from other V2V links; is the binary spectrum allocation factor, representing the sub-channel allocation of the V2I link; is the interference of the kth V2I link to the kth V2V link; is the interference of the k′th V2V link transmitter to the kth V2V link receiver.

4. A V2X communication and perception joint resource allocation method according to claim 1, characterized in that: In the NR-V2X communication environment in S3, cyclic prefix OFDM signals are used for communication and perception.

5. A V2X communication and perception joint resource allocation method according to claim 1, characterized in that: The state input of the reinforcement learning model in S4 is defined as the channel gain of all V2I links, the channel gain of V2V links and the interference between links, and its state input expression is: s t ={g(t),I l (t),I k (t)}, Where g(t) represents the channel gain and interference channel of all V2I links and V2V links, I l (t), represents the interference power of the first V 2I link, I k (t) represents the interference power of the kth V 2V link, represents the kth V2V channel gain, represents the interference channel from the kth V2V transmitter to the kth V2V receiver on the nth subchannel, represents the interference channel from the lth V2I transmitter to the kth V2V receiver on the nth subchannel, represents the interference channel from the kth V2V transmitter to the BS on the lth subchannel, represents the channel power gain from the transmitter of the lth V2I link to the nth subchannel.

6. A V2X communication and perception joint resource allocation method according to claim 1, characterized in that: The action space of the reinforcement learning model in S4 is defined as the sub-channel allocation and power allocation of all V2V links, and its specific expression is: a t ={a1(t),a2(t)}, a1(t) represents the allocation of subchannels; Indicates whether the kth V2V link reuses the lth subchannel, where indicates that the lth subchannel is assigned to the kth V2V link, otherwise a2(t) represents the power distribution, represents the power of the kth V2V link on the lth subchannel.

7. A V2X communication and perception joint resource allocation method according to claim 1, characterized in that: The reward of the reinforcement learning model in S4 is defined as the total communication rate of the V2I link and the perception accuracy of the V2V positioning; The above mentioned reward also takes into account the minimum SINR constraints of the V2I link and the V2V link, and the specific expression of the constraint is: In the formula, Constrain its V2I communication rate to be maximized; Constrain its CRLB limit to be minimized, that is, the perceived V2V positioning accuracy is maximized; and The constraint is to maximize the SINR constraints on V2I and V2V; L and K represent the number of V2I links and V2V links; a(t) and b(t) represent the satisfaction of the QoS constraints of all V2I links and V2V links, where a l (t) indicates whether the lth V2I link meets the minimum SINR requirement. a l (t)=1, otherwise, a l (t) = 0; similarly b k (t) indicates whether the kth V2V link can meet the minimum SINR requirement. b k (t) = 1, otherwise b k (t)=0.

8. A V2X communication and perception joint resource allocation method according to claim 1, characterized in that: The CRLB limit perceived in S3 is calculated as follows: where u s (t) is the CRLB of the positioning estimation error, SNR is the signal-to-interference-noise ratio perceived by the vehicle, and P T (t) is the transmit power of the V2V link, f c (t) is the carrier frequency, G is the antenna gain, σ is the radar cross section RCS, P n is the noise power, k B is the Boltzmann constant, T0 is the reference temperature, F is the receiver noise, and d is the distance between the target vehicle detected by the sensor and the transmitting vehicle.

9. A V2X communication and perception joint resource allocation method according to claim 1, characterized in that: The expression of the overall optimization objective of optimizing the network using the MADDPG reinforcement learning algorithm in S4 is: Where u(t) is the overall index of synaesthesia integration, u c (t) is the communication utility index, α u , β u is the proportionality coefficient, u s (t) is the perceived utility index.

10. A V2X communication and perception joint resource allocation method according to claim 1, characterized in that: The multiple service quality constraints in S4 include constraints C1 to C9, where constraint C1 means that each V2I link can only use one subchannel, and each subchannel can be occupied by at most one V2I link; constraint C2 means that each V2V link can reuse the subchannel resources of at most one V2I link, but each subchannel can be reused by multiple V2V links; constraints C3 and C4 mean that the power of the V2I link and the V2V link cannot exceed the maximum power limit and Constraint C5 indicates that the power of the V2I link and the V2V link cannot be less than 0; Constraints C6 and C7 indicate that the SINR of the V2I link and the V2V link cannot be lower than the minimum SINR threshold. and Constraint C8 represents the Cramer-Rao lower bound for V2V Sidelink positioning estimation The minimum positioning accuracy requirement τ cannot be exceeded; constraint C9 indicates that the entire network resources are divided into two parts: communication resources and perception resources; the expressions of each constraint are as follows: C9:N c +N s ≤N total In the formula, is the binary spectrum allocation factor, representing the sub-channel allocation of the V2I link; L is the number of V2I links indicates that the nth subchannel is assigned to the kth V2V link, otherwise K is the number of V2V links is the transmission power of the lth V2I link; is the transmission power of the kth V2V link; is the received signal-to-interference-noise ratio of the lth V2V link; is the received signal-to-interference-noise ratio of the kth V2I link; and is the minimum threshold of the received signal-to-interference-noise ratio; is the Cramer-Rao lower bound of the distance d estimation error; τ is the minimum positioning accuracy requirement; N total is the overall resource; N c N is the communication resource; s For perception resources.

Images