A Blind Retransmission Strategy Optimization Method Based on Deep Reinforcement Learning in V2V Communication

Abstract

This invention belongs to the field of vehicle networking technology, and particularly relates to a blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication. The method includes: observing the vehicle to calculate the packet reception rate (PRR) at different observation distances based on received perception messages, fitting a curve showing the relationship between PRR and distance, and estimating the effective communication range (ETR) based on this curve; then calculating the net busy rate (NCBR) of the channel based on the signal-to-noise ratio (SNR) of the received data packets; comprehensively considering ETR and NCBR, constructing a blind retransmission strategy optimization model and transforming it into a reinforcement learning problem, designing the corresponding state space, action space, and reward function; and training the vehicle node as an agent using a dual deep Q-network algorithm to dynamically obtain the optimal number of blind retransmissions, optimize the effective communication distance and the net busy rate of the channel, and achieve adaptive adjustment of the blind retransmission strategy, thereby fully utilizing channel resources and improving the reliability of V2V communication.

Description

Technical Field

[0001] This invention belongs to the field of vehicle-to-everything (V2X) technology, specifically a blind retransmission strategy optimization method based on deep reinforcement learning in vehicle-to-vehicle (V2V) communication, which is applied to V2V broadcast communication of the IEEE 802.11p / 802.11bd standard. Background Technology

[0002] With the rapid development of Intelligent Transportation Systems (ITS), Vehicle-to-Everything (V2X) communication has received widespread attention as one of the core supporting technologies for improving road safety, traffic efficiency, and intelligence. In particular, within the V2X communication architecture, direct vehicle-to-vehicle (V2V) communication enables real-time exchange of vehicle status information without infrastructure dependence, which is crucial for scenarios such as accident warning, lane changing, and traffic cooperation.

[0003] To ensure the reliability and real-time performance of V2V communication, the IEEE 802.11p standard and its upgraded version, IEEE 802.11bd, both introduced a "blind retransmission" mechanism. This mechanism allows the sending node to retransmit the same data packet up to three times without receiving confirmation from the receiving node. The receiving node then integrates the multiple received copies using Maximum Ratio Combining (MRC) technology, thereby improving the data decoding success rate. Because this mechanism requires no feedback confirmation in a broadcast environment, reduces the control signaling load, and has the advantages of simple structure and low latency, it has become a standard means of ensuring the reliability of V2V communication.

[0004] However, with the increasing deployment density of V2V systems, especially in high-traffic and high-speed scenarios, blind retransmission mechanisms have revealed numerous bottlenecks. The core issue is that, given limited channel resources, the repeated data transmission introduced by blind retransmission significantly exacerbates channel congestion, leading to increased packet collision rates and consequently reducing the overall communication performance of the system. Furthermore, traditional blind retransmission strategies mostly employ static parameter settings, such as a fixed number of retransmissions, lacking real-time perception and adaptation to key parameters such as channel conditions, vehicle density, and motion characteristics. This fails to meet the coordinated demands for communication reliability and efficiency in highly dynamic traffic environments.

[0005] To address the aforementioned issues, existing technologies have proposed several improvement methods to enhance the adaptability and performance of V2V blind retransmission mechanisms in complex scenarios. For example, Chinese patent document CN113411779B discloses a design method and apparatus for maximizing user capacity in vehicle-to-everything (V2X) networks while ensuring reliability. This method is primarily based on the NR-V2X architecture and achieves a dynamic balance between user capacity and reliability by allocating time-frequency resources, calculating link reliability, and channel occupancy status. This scheme establishes a mapping relationship between link quality and user concurrency capability at the resource allocation level, aiming to improve the overall network's concurrent communication capability. However, its core focus is on maximizing the overall network's user capacity. It enhances the link reliability of message propagation between vehicles in the system by limiting the number of vehicles simultaneously sending messages, reducing the number of message retransmissions to zero. Its relationship with the reliability control of the blind retransmission mechanism itself in V2V communication is relatively indirect.

[0006] Patent CN115765935B proposes a Hybrid Automatic Repeat Request (HARQ) method and apparatus. In this method, the transmitting end obtains the number of consecutive transmissions of data packets based on channel parameters using a HARQ strategy model, and then sends data packets according to this value. This effectively improves the probability of successful data reception and minimizes the resource consumption of the communication system. Compared to traditional blind retransmission mechanisms, this method requires the receiving end to determine whether a data packet has been successfully received and the number of retransmissions upon successful reception, consuming certain resources, especially increasing the communication load under high density. Furthermore, this scheme is suitable for unicast scenarios using feedback mechanisms and does not strictly meet the requirements of V2V broadcast scenarios.

[0007] Overall, although the two existing technologies mentioned above have made some progress in improving link reliability or resource utilization efficiency, their technical solutions suffer from the following common shortcomings, making it difficult to effectively address the core challenges of V2V communication systems in complex traffic environments:

[0008] First, blind retransmission strategies lack dynamic adaptability. Existing methods mostly employ fixed or static threshold-driven retransmission strategies, which are difficult to respond to drastic changes in channel conditions in real time. For example, instantaneous changes in vehicle density, non-line-of-sight propagation environments, and fast-fading channels can all lead to a serious mismatch between the traditional retransmission count setting and the actual reception probability, resulting in resource waste or transmission failure.

[0009] Second, the control mechanism neglects node fairness. In high-load V2V networks, some nodes occupy channel resources for extended periods due to high retransmission strategies, forming a "transmission monopoly," while the access success rate and latency performance of other nodes deteriorate significantly. Existing technologies rarely involve joint modeling and control based on channel state and node policy fairness, making it difficult to guarantee the balance and sustainability of the overall system performance.

[0010] Third, there is a lack of intelligent optimization capabilities to cope with dynamic and non-stationary environments. V2V communication environments exhibit significant non-stationarity, with vehicle topology and channel conditions changing rapidly over time. Traditional control methods based on static models or rules are prone to failure in this context. Although some literature has introduced reinforcement learning models, their state-space design is often overly simplified, failing to fully incorporate the joint description of key physical layer characteristic parameters (such as ETR and NCBR). Existing technologies often treat these parameters in isolation, failing to construct a unified collaborative decision-making model. This results in optimization results that are overly sensitive to specific scenarios, lacking universality and stability.

[0011] Therefore, while existing blind retransmission mechanisms in V2V communication systems play a role in ensuring communication reliability, their strategy design has significant limitations when facing high-density, highly dynamic traffic environments. How to dynamically sense and integrate multi-dimensional communication parameters, adaptively adjust the number of blind retransmissions, and balance communication efficiency and channel load while ensuring transmission reliability has become a core problem that urgently needs to be solved in this technical field. Summary of the Invention

[0012] In view of this, the present invention aims to solve the technical problems of existing blind retransmission strategies in V2V communication, such as lack of adaptability, inability to dynamically perceive channel status, waste of retransmission resources, and channel congestion. It provides a blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication. By constructing a reinforcement learning model with the Effective Communication Range (ETR) and Net Busy Rate (NCBR) as state inputs, and using a dual deep Q-network algorithm to dynamically adjust the number of blind retransmissions, the retransmission decision is optimized in real time in complex traffic environments, effectively improving the reliability and channel utilization efficiency of V2V communication.

[0013] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0014] A method for optimizing blind retransmission strategies based on deep reinforcement learning in V2V communication includes:

[0015] S1: The observation vehicle calculates the packet reception ratio (PRR) at different observation distances r based on the received perception messages. The observation vehicle is any vehicle participating in V2V communication. The packet reception ratio PRR is the ratio of the number of data packets successfully received by the observation node to the total number of data packets sent by the neighboring node. The packet reception ratio PRR is a function of the reception distance. The observation vehicle is the receiving node, the neighboring node is the sending node, and the observation distance is the reception distance between the sending node and the receiving node.

[0016] S2: Based on the packet reception rate (PRR) at different receiving distances, the PRR-r relationship curve is obtained by polynomial fitting, and then the effective transmission range (ETR) is estimated. The effective transmission range (ETR) is the communication distance that satisfies the preset packet reception rate (PRR) threshold.

[0017] S3: Calculate the Net Channel Busy Ratio (NCBR) under the blind retransmission mechanism based on the signal-to-interference-plus-noise-ratio (SINR) of the received signal;

[0018] S4: Construct a blind retransmission strategy optimization problem based on the effective communication range (ETR) and the net busy rate (NCBR) of the channel, and map the optimization problem into a deep reinforcement learning problem. Design the corresponding state space, action space and reward function. The state space includes at least the effective communication range (ETR) and the net busy rate (NCBR) of the channel. The action space includes different blind retransmission times. The reward function considers both communication reliability and channel load.

[0019] S5: Using vehicle nodes as reinforcement learning agents, the optimization problem is solved by employing the Double Deep Q-learning Network (DDQN) algorithm to dynamically obtain the optimal number of blind retransmissions, which is then applied to subsequent data broadcasting.

[0020] In one example of this application, the packet reception rate PRR is the rate at which the observation node receives packets during the observation period T. ob The number of data packets successfully received within the observation period T is compared with the number of neighboring nodes. ob The ratio of the total number of data packets sent within the period, where:

[0021]

[0022] Where N is the number of neighboring nodes, For the observation node in the observation period T ob The number of data packets successfully received from neighbor node i. Let λ be the total number of data packets sent by node i during the observation period. The number of packets sent per second by adjacent nodes can be considered to be the same, denoted as λ, and the unit is Hertz. Approximately λT ob .

[0023] In one example of this application, the neighboring nodes are grouped into different observation distance intervals according to their distance d from the observing node. The observing node calculates the average packet reception rate of L neighboring nodes in each distance interval, which is taken as the PRR at the observation distance r, denoted as PRR(r):

[0024]

[0025] k is the index of the observed node within the observation distance interval.

[0026] In one example of this application, in step S2, the packet reception rate (PRR) at different observation distances r is calculated based on step S1, and a PRR-r curve is obtained using a polynomial fitting method. The effective communication range (ETR) is defined as the packet reception rate (PRR) at different observation distances being greater than a preset threshold PRR. th The maximum communication distance, i.e.

[0027]

[0028] in, To satisfy PRR ≥ PRR th The effective propagation distance, and the packet reception rate preset threshold PRRth is the given packet reception rate PRR threshold.

[0029] In one example of this application, in step S3, the net busy rate (NCBR) of the channel is defined as the signal-to-noise ratio (SINR) of the first data packet in a series of retransmitted data packets exceeding the SINR threshold γ. th Time allocation:

[0030]

[0031] Among them, T first The first data packet in a series of retransmitted packets to have a signal-to-noise ratio (SINR) exceeding the SINR threshold γ. th The time occupied by the data packets, T obs Observation time, each node every T obs Update the Net Busy Rate (NCBR) of the channel.

[0032] In one example of this application, in step S3, the signal-to-noise ratio (SINR) value γ of the j-th data packet signal is... j Establish the following model:

[0033]

[0034] Among them, P rj I represents the received signal power; Noise represents the average noise power; I avj This represents the average interference power received by the receiving node during the reception of the j-th data packet.

[0035] In one example of this application, the average interference power I avj The weighted average of the interference signal strengths within the data packet reception period is obtained, and the calculation formula is shown below:

[0036]

[0037] Among them, t p N represents the data packet transmission time. i，j Let t be the number of time segments divided during the transmission process of nodes i and j, and t be the number of time segments in the nth time segment. n The internal interference power always remains constant. For time period t n The set of signals that interfere with receiving node j, ξ i,k ∈[0,1] represents the proportion of frequency overlap between the interference signal emitted by node k and the useful signal.

[0038] In one example of this application, in step S4, the state space is defined by the current effective propagation distance. A two-dimensional state vector composed of the channel net busy rate (NCBR). t represents the current time, and the action space is the set a of blind retransmission counts. t = {0, 1, 2, 3}, and the reward function is r t =w1·R Range +w2·R CBR Where w1 and w2 are weighting coefficients, satisfying w1 + w2 = 1. NCBR max This represents the maximum possible net channel busy rate.

[0039] In one example of this application, in step S5, the dual-deep Q-network algorithm includes constructing a main Q-network and a target Q-network, which are used to estimate the action value function and provide a stable target value to reduce oscillations and instabilities during training, respectively. The Q-value is trained through deep learning until convergence, and the current optimal number of blind retransmissions is selected using the trained Q-value function.

[0040] In one example of this application, in step S5, the main Q-network and the target Q-network adopt a multi-layer fully connected neural network, and the input two-dimensional state vector s t Output four Q-values ​​corresponding to the number of blind retransmissions. The parameters of the primary Q-network are θ and the parameters of the target Q-network are θ. - The Q-value update formula is:

[0041]

[0042] Where α is the learning rate, β is the discount factor, and r tFor in s t Take action a in the state t The reward obtained later, Q old This represents the Q-value predicted by the online Q-network. This represents the Q-value predicted by the target Q-network. During training, an ε-greedy strategy is used for action selection, and the loss function is optimized based on temporal difference (TD) error.

[0043]

[0044] Where D represents the set of state-action pairs, y Q This represents the expected target Q value. The expression is as follows:

[0045] Compared with existing technologies, the blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication described in this invention has the following advantages:

[0046] 1. The blind retransmission optimization method proposed in this application dynamically estimates the effective communication range (ETR) by constructing a packet reception rate (PRR) statistical mechanism that refines distance grouping and time window. Combined with signal-to-noise ratio (SINR) modeling of the received signal, it accurately calculates the net busy rate (NCBR) of the channel. This enables the system to accurately reflect the communication status of the vehicle under different locations and channel loads, allowing the blind retransmission strategy to be continuously adjusted according to changes in the actual communication environment. It no longer relies on a fixed retransmission number setting, effectively avoiding channel congestion caused by excessive retransmission or data packet loss caused by insufficient retransmission. It ensures stable communication performance under different traffic densities, propagation paths, and noise environments, significantly improving the adaptability of the vehicle network system to complex dynamic environments and the accuracy of the retransmission strategy.

[0047] 2. This application utilizes the meticulous design of state space, action space, and reward function in deep reinforcement learning to construct a reinforcement learning problem model. The Effective Communication Range (ETR) and Net Channel Busyness Rate (NCBR) are jointly modeled as state vector inputs, and the number of blind retransmissions is enumerated as an action set. A dynamic weighting mechanism is introduced to balance rewards for communication distance and channel load. The system continuously senses the communication environment and iteratively optimizes the retransmission strategy after receiving feedback. This enables the learning model to find a retransmission decision path that balances reliability and channel efficiency under limited resources and complex interference. Under the reinforcement learning structure, online learning and dynamic adjustment of the blind retransmission strategy are achieved. Simultaneously, the separation of the target Q-network and the main Q-network reduces the risk of overestimation. Combined with experience replay and an ε-greedy strategy, the generalization ability and exploration efficiency of the strategy are improved, thereby achieving higher strategy stability and deployment adaptability.

[0048] 3. This application utilizes a reinforcement learning-driven blind retransmission strategy optimization scheme to enable vehicle nodes to make real-time decisions on the optimal number of blind retransmissions during communication. This improves channel utilization while ensuring communication reliability under the influence of the blind retransmission mechanism. Compared to traditional schemes, this approach enhances the overall resource utilization efficiency and stability of the system, making it particularly suitable for typical V2V communication scenarios such as high-speed vehicles and frequent channel fluctuations in urban roads and intersections. Attached Figure Description

[0049] Figure 1 This is a flowchart of the blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication according to an embodiment of the present invention;

[0050] Figure 2 The relationship between the number of blind retransmissions, effective communication range, and net busy rate (NCBR) of the channel under the Winner+B1 channel model;

[0051] Figure 3 This relates the number of blind retransmissions to the effective communication range and the net busy rate (NCBR) of the channel under the ECC channel model. Detailed Implementation

[0052] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0053] In the description of this application, it should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and devices should be considered part of the specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters in the following figures denote similar items; therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0054] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and are not limited in number; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0055] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0056] like Figure 1 As shown, this application discloses a blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication, including:

[0057] S1: The observation vehicle calculates the packet reception ratio (PRR) at different observation distances r based on the received perception messages. The observation vehicle is any vehicle participating in V2V communication. The packet reception ratio (PRR) is the ratio of the number of data packets successfully received by the observation node to the total number of data packets sent by the neighboring node. The packet reception ratio (PRR) is a function of the reception distance. The observation vehicle is the receiving node, the neighboring node is the sending node, and the observation distance is the reception distance between the sending node and the receiving node.

[0058] S2: Based on the packet reception rate (PRR) at different observation distances r, the PRR-r relationship curve is obtained by polynomial fitting. Then, the effective communication range (ETR) is estimated. The effective communication range (ETR) is the communication distance that satisfies the preset packet reception rate (PRR) threshold.

[0059] S3: Calculate the Net Channel Busy Ratio (NCBR) under the blind retransmission mechanism based on the signal-to-noise-interference ratio (SINR) of the received signal;

[0060] S4: Construct a blind retransmission strategy optimization problem based on the effective communication range (ETR) and the net busy rate (NCBR) of the channel, and map the optimization problem into a deep reinforcement learning problem. Design the corresponding state space, action space and reward function. The state space includes at least the effective communication range (ETR) and the net busy rate (NCBR) of the channel. The action space includes different blind retransmission times. The reward function considers both communication reliability and channel load.

[0061] S5: Using vehicle nodes as reinforcement learning agents, the optimization problem is solved by employing the Double Deep Q-learning Network (DDQN) algorithm to dynamically obtain the optimal number of blind retransmissions, which is then applied to subsequent data broadcasting.

[0062] This application discloses a blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication. It constructs state variables reflecting communication quality and channel load: Effective Communication Range (ETR) and Net Channel Busyness Rate (NCBR), and uses these as input states for reinforcement learning strategy decision-making to achieve adaptive adjustment of the number of blind retransmissions. Specifically, the vehicle first receives perception messages broadcast from the surrounding environment (such as Basic Safety Message (BSM)). The system calculates the packet reception rate (PRR) at different observation distances (r) and obtains the PRR-r curve using polynomial fitting. Then, it estimates the effective communication range (ETR). Based on the signal-to-noise ratio (SINR) of the received signal, it determines the net busy rate (NCBR) under the blind retransmission mechanism. The ETR and NCBR are then combined to form the state input of the reinforcement learning model. Different retransmission count options are enumerated through a predefined action space, and a reward function that balances communication reliability and channel occupancy control is set to guide the learning process towards improving transmission success rate while avoiding excessive channel resource consumption. Finally, each vehicle node acts as a reinforcement learning agent, and a dual-deep Q-network algorithm is used for state-action mapping optimization. After training convergence, the optimal number of blind retransmissions is quickly determined based on the current state in real-time communication and used in subsequent broadcasts. The overall strategy balances adaptability, efficiency, and system fairness, making it suitable for high-density, high-dynamic vehicle-to-everything (V2X) communication scenarios. The blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication described in this application is based on the core principle of utilizing the "trial and error + reward and punishment" mechanism in reinforcement learning. By using the Effective Communication Range (ETR) and Net Channel Busy Rate (NCBR) as input states for the learning model, the vehicle can accurately perceive the current communication environment and output the most suitable number of blind retransmissions according to the continuously optimized strategy function during the learning process. During the training phase, the system records feedback information such as the success or failure of communication and the channel occupancy level after adopting a certain retransmission strategy in each state, and converts it into a reward value to back-correct the Q-value estimation. Through a dual Q network, the action value function is estimated and a stable target value is provided, avoiding overfitting and overestimation, improving the stability and generalization ability of learning, and finally forming a set of optimal retransmission strategy mappings. During the communication execution phase, the vehicle node perceives the Effective Communication Range (ETR) and Net Channel Busy Rate (NCBR) in real time and inputs them into the learned strategy network, quickly outputting the most suitable number of retransmissions for the current environment, realizing the closed-loop control capability of "optimizing while communicating".

[0063] The blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication described in this application, compared with the traditional fixed retransmission strategy, improves communication reliability and channel utilization by comprehensively utilizing ETR, while suppressing channel congestion caused by high-frequency retransmissions through NCBR. It maintains stable reception performance even under high overall system load, significantly reducing packet loss rate and channel collision probability. Furthermore, dynamically adjusting the number of blind retransmissions allows vehicles to better adapt to complex communication environments, reducing unnecessary retransmissions and lowering communication latency and energy consumption. In addition, this strategy possesses online real-time decision-making capabilities, enabling rapid response to dynamic changes in vehicle density and channel conditions, effectively reducing communication latency and energy consumption, improving resource utilization efficiency, and exhibiting good policy transfer capabilities and deployment feasibility under the support of reinforcement learning structures.

[0064] As a preferred example of this application, in step S1, the packet reception rate PRR is the rate at which the observation node receives packets during the observation period T. ob The number of data packets successfully received within the observation period T is compared with the number of neighboring nodes. ob The packet reception rate (PRR) is the ratio of the number of packets successfully received by an observer node within the observation period to the total number of packets sent by the neighbor node within the observation period. In V2V scenarios, the PRR is typically defined as a function of the reception distance *r*. It is estimated by the ratio of the number of packets successfully received by an observer node from its neighbor node within the observation period to the total number of packets sent by the neighbor node within the observation period. The packet sending rates of neighboring nodes can be considered to be uniform, both being λ. Therefore, the packet sending rate of neighbor node *i* within the observation period *T* is... ob The number of packets sent internally can be approximated as λT ob The vehicle can analyze the distance D to each neighbor node i based on the received perception messages. i Based on the number of messages received, an approximate packet reception rate (PRR) is calculated as follows:

[0065]

[0066] in For the observation node in the observation period T ob The number of data packets successfully received from neighbor node i. λ represents the total number of data packets sent by neighbor node i during the observation period, and λ is the number of packets sent per second, in Hertz.

[0067] This method, based on statistical observation principles, transforms the success rate of inter-vehicle communication reception into a quantifiable probability index, Packet Reception Rate (PRR), and sets it as a function of distance *r* to account for the impact of distance on reception quality. The method further utilizes a set observation period *T*. obThe observation node counts the number of perceived messages received from each neighboring node within the specified period and estimates the actual number of messages sent by each neighboring node based on their broadcast frequency, thereby calculating the Packet Reception Rate (PRR) value. In the example of this application, the observation period T... ob The system is dynamically adjusted based on real-time communication requirements and channel change rate to address statistical errors caused by rapid channel changes.

[0068] As a preferred example of this application, in step S1, the neighboring nodes are grouped into different observation distance intervals according to their distance d from the observing node, corresponding to different observation distances r. The observing node calculates the average packet reception rate of L neighboring nodes in each distance interval as the PRR at the observation distance r, denoted as PRR(r). In the example of this application, the packet reception rate PRR is a function of the observation distance r, given m observation distances [r1, ..., r]. j , ..., r m ], where j (1≤j≤m) is the index of the observation distance. R represents the communication range calculated based on the channel fading model and path loss, and Δr represents the observation distance increment. Neighboring nodes are grouped by distance into corresponding observation distance intervals. The average packet reception rate PRR(r) of L nodes belonging to the same observation distance interval is calculated as follows:

[0069]

[0070] k is the index of the observed node within the observation distance interval.

[0071] To further improve the spatial resolution and communication environment awareness of packet reception rate estimation, this application groups all neighboring nodes according to their distance from the receiving node, and generates a series of observation distance intervals based on a set interval Δr. Each interval contains several neighboring nodes that are close to the receiving node. The receiving node observes the data within a given observation period T. ob The system records the number of sensing messages sent and received by each neighboring node, calculates its individual packet reception rate (PRR) value, and then averages the PRR of all nodes belonging to each interval to obtain the corresponding distance r. jThe average packet reception rate (PRR(r)) is calculated by grouping neighboring nodes according to their reception distance. This method makes the PRR estimation result more closely reflect the actual spatial attenuation characteristics, overcoming the problem that traditional average statistical methods cannot reflect the impact of distance changes on communication. This gives the system a stronger ability to characterize communication performance under different distance conditions, thereby improving the accuracy and timeliness of distance-based decision-making in blind retransmission strategies. Especially in the dynamically changing vehicle network environment, with uneven vehicle distribution density and complex propagation conditions, this method can more effectively screen out distance areas where communication quality deteriorates. This allows subsequent strategies to adjust the number of retransmissions in a targeted manner to cope with long-distance communication distortion. At the same time, after introducing the observation distance interval Δr and the communication range R, this mechanism can flexibly set the segment granularity according to system requirements, improving estimation accuracy while taking into account resource consumption, achieving a dynamic balance between accuracy and performance. This provides a more practical and engineering-applicable input basis for V2V communication strategy optimization.

[0072] As a preferred example of this application, in step S2, based on the packet reception rate (PRR) calculated in step S1, the effective communication range (ETR) is defined as the packet reception rate (PRR) value of the observation node at different observation distances being greater than a preset packet reception rate threshold (PRR). th The maximum communication distance, i.e.:

[0073]

[0074] in, To satisfy PRR ≥ PRR th Effective propagation distance, packet reception rate preset threshold (PRR) th Given the packet reception rate (PRR) threshold.

[0075] This application proposes an effective communication range (ETR) calculation method based on a packet reception rate threshold. This method obtains the average packet reception rate (PRR(r)) at different observation distances r and compares it with a preset packet reception rate threshold (PRR). th By comparison, it is determined that PRR ≥ PRR th The maximum receiving distance under the given conditions is taken as the effective communication range (ETR), and the packet reception rate is set to a preset threshold (PRR). th To reflect the distance boundary for achieving a high reliability standard in communication, the receiving node combines the average PRR(r) data obtained by distance grouping in step S1 above, and sequentially retrieves the average reception rate for each observed distance interval. Once it is detected that the packet reception rate PRR value of a certain interval is lower than the preset packet reception rate threshold PRR for the first time... thIf the observed distance *r* that satisfies the previous condition is recorded as the effective communication range (ETR), then this method essentially transforms communication quality into a judgment of the relationship between spatial variables and a threshold. It quantifies the current channel reachability through simple logic, featuring simple algorithm and high computational efficiency. It can quickly extract key communication capability indicators and is particularly suitable for the frequently updated policy input requirements in dynamic vehicle-to-everything (V2X) environments. It is suitable for deployment on in-vehicle terminals for rapid response and real-time updates, effectively improving policy adaptation accuracy and reducing system computational burden. In the example of this application, the packet reception rate preset threshold (PRR) is... th Set to 0.9.

[0076] As a preferred example of this application, in step S3, the net busy rate (NCBR) is defined as the signal-to-noise ratio (SINR) of the first data packet in a series of retransmitted data packets exceeding the SINR threshold γ. th Time allocation:

[0077]

[0078] Among them, T first The first data packet in a series of retransmitted packets to have a signal-to-noise ratio (SINR) exceeding the SINR threshold γ. th The time occupied by the data packets, T obs Observation time, each node every T obs Update the Net Busy Rate (NCBR) of the channel.

[0079] In the example of this application, to accurately reflect the quality characteristics of channel occupancy under the blind retransmission mechanism, this application takes the first data packet in the multiple blind retransmission data packets as the analysis object, and uses its signal-to-noise ratio (SINR) as the criterion for whether the channel is being used effectively. When the observation node receives the signal from the multiple retransmissions, it observes the signal for a preset observation time T. obs Within this timeframe, the transmission time occupied by the first data packet that satisfies the signal-to-noise ratio (SINR) is accumulated to T. first to compare it with T obs The formation ratio is the Net Channel Busyness Rate (NCBR) value, which indicates the degree to which the channel is effectively occupied by high-quality communication within a given time window. This value is calculated every T... obs Reassess the Net Channel Busyness Rate (NCBR) to achieve continuous awareness of channel status.

[0080] This application introduces a signal-to-noise ratio threshold γ. th The mechanism of limiting the observation time window makes the assessment of the channel net busy rate (NCBR) more targeted and real-time, and the signal-to-noise ratio (SNR) threshold γ th The observation time T serves as a standard value for evaluating the decodeable strength of a signal. obsThe scope ensures that the calculations have sufficient statistical significance without incurring additional resource burdens, effectively balancing observation accuracy and computational efficiency, while each node operates at a T... obs The system automatically updates the Net Busy Rate (NCBR) value periodically, enabling it to obtain real-time channel utilization quality status. This provides continuous input for dynamic optimization of blind retransmission strategies, resulting in a more reasonable allocation of retransmission times, reducing resource waste caused by redundant retransmissions, and improving communication reliability and system responsiveness. This is particularly suitable for highly dynamic and resource-constrained V2V network scenarios. In the example of this application, the signal-to-noise ratio (SNR) threshold γ... th The value is usually set to -85dBm, the net busy rate (NCBR) ranges from [0, 1], and the observation time T is... obs The value range of T is 50ms to 500ms. As a preferred value, T... obs Typically, 100ms is used.

[0081] As a preferred example of this application, in step S3, the signal-to-noise ratio (SINR) value γ of the j-th data packet signal is... j Establish the following model:

[0082]

[0083] Among them, P rj I represents the received signal power; Noise represents the average noise power; I avj This represents the average interference power received by the receiving node during the reception of the j-th data packet.

[0084] In the example of this application, since the interval between each retransmitted data packet signal and the previous signal is a Short Interframe Space (SIFS), the transmission quality of the data packet is evaluated by identifying the first packet of each data packet during blind retransmission and constructing a signal-to-noise-interference ratio (SINR) model based on the SIFS structure. In this model, the received signal power P... rj The average noise power is calculated based on the actual distance between vehicles and the transmission power, reflecting the signal attenuation at different distances. The average noise power is dynamically collected and averaged based on the noise level in the environment where the receiving node is located, avoiding errors caused by static noise assumptions. The average interference power I... avj Based on the data packet reception time, the system is divided into multiple fixed-length time periods. Within each time period, other signal sources that simultaneously interfere with the receiving node are identified. These sources are then weighted according to their transmission intensity and spectral overlap and included in the overall interference power statistics. This mechanism can not only accurately calculate the signal-to-noise ratio (SINR) of the first data packet, but also update it in real time in the frequently changing vehicle network environment, effectively supporting the low-level communication quality assessment required for blind retransmission strategy optimization.

[0085] In the example of this application, the average interference power I avj The weighted average of the interference signal strengths within the data packet reception period is obtained, and the calculation formula is shown below:

[0086]

[0087] Among them, t p N represents the data packet transmission time. i，j Let t be the number of time segments divided during the transmission process of nodes i and j, and t be the number of time segments in the nth time segment. n The internal interference power always remains constant. For time period t n The set of signals that interfere with receiving node j, ξ i，k ∈[0,1] represents the ratio of frequency overlap between the interference signal emitted by node k and the useful signal (0 represents no overlap, 1 represents complete overlap).

[0088] The signal-to-noise ratio (SINR) modeling method described in this application effectively improves the accuracy and adaptability of signal quality assessment by introducing dynamic calculation methods based on vehicle positioning and environmental perception. Compared with traditional static parameter models, this scheme can comprehensively analyze the reception quality of each data packet based on the vehicle's real-time location, transmission power, noise changes, and surrounding interference, making the SINR assessment closer to the actual communication environment. This provides a reliable basis for effective packet identification and resource scheduling in blind retransmission strategies. Specifically, by accurately distinguishing between high and low SINR, it helps to identify redundant retransmissions or misjudged packet loss, thereby reducing unnecessary retransmissions, alleviating channel load, improving overall communication efficiency and reliability, and reducing the risk of channel congestion.

[0089] As a preferred example of this application, in step S4, the state space is defined by the current effective propagation distance. A two-dimensional state vector composed of the channel net busy rate (NCBR). t represents the current time, and the action space is the set a of blind retransmission counts. t = {0, 1, 2, 3}, and the reward function is r t =w1·R Range +w2·R CBR Where w1 and w2 are weighting coefficients, satisfying w1 + w2 = 1. NCBR max This represents the maximum possible net channel busy rate.

[0090] In the example of this application, to realize the effective application of reinforcement learning in the optimization of blind retransmission strategy in vehicle-to-everything (V2X) networks, this application designs a reinforcement learning state modeling scheme for communication reliability and channel load control in step S4. This scheme includes three main parts: state space, action space, and reward function. The state space is a two-dimensional vector composed of the effective propagation distance at the current time t and the net busy rate (NCBR) of the channel. This indicates that the input from the intelligent agent's perception of the current environment... The effective transmission distance (ETD) represents the quality performance of the current communication range. The NCBR characterizes the efficiency of current channel resource utilization. The action space corresponds to the number of blind retransmissions N. rep , which is the following set:

[0091] a t ={0, 1, 2, 3}

[0092] Among them, a t =0 indicates no retransmission, only one transmission; a t =1 indicates one retransmission, for a total of two transmissions; a t =2 indicates 2 retransmissions, for a total of 3 transmissions; a t =3 indicates 3 retransmissions, for a total of 4 transmissions. This represents a strategy space with four options: no retransmission to a maximum of 3 retransmissions. The model selects one strategy at a time, executes the corresponding retransmission action, and observes the system feedback, thus updating the strategy. And NCBR value, to ensure flexible system decision-making, the reward function integrates two aspects of indicators: communication quality and channel load, with r t =w1·R Range +w2·R CBR The formal expression shows that the weight coefficients w1 and w2 can be dynamically adjusted according to the priority of the current vehicle network communication service. The reinforcement learning agent learns through continuous interaction and iteration, and gradually optimizes the blind retransmission number selection strategy under different communication conditions. In order to maximize the long-term cumulative reward value, it achieves a dual balance between communication reliability and channel efficiency, and dynamically adapts to the strategy requirements of different load densities and business scenarios.

[0093] This application will cover the effective propagation distance. Using the Net Channel Busyness Rate (NCBR) as a state input variable enables the model to comprehensively perceive the coverage capability and channel utilization of the current communication environment, avoiding the one-sidedness of a strategy driven by a single indicator. The effective transmission distance is comprehensively considered in the reward function design. The Net Channel Busyness Rate (NCBR) incentivizes high reliability and penalizes channel overload, ensuring efficient channel utilization. A weighted mechanism adjusts the optimization focus on communication reliability and channel efficiency based on the type of vehicle-to-everything (V2X) task and real-time communication requirements. This ensures appropriate retransmission strategies are matched for both emergency safety message transmission and resource-intensive data service transmission. This mechanism effectively prevents channel congestion caused by excessive retransmissions and avoids reception failures due to insufficient retransmissions, improving the overall system communication success rate and spectrum utilization. Under the above design guidance, the reinforcement learning model can quickly converge to the optimal strategy, improving the reliability of V2X communication, especially demonstrating significant stability and reliability improvements in complex road scenarios with high vehicle density and rapid channel changes. In the example of this application, NCBR... max =1,R CBR ∈[0,1],NCBR t The smaller the value, the closer the reward is to 1.

[0094] As a preferred example of this application, in step S5, the dual-deep Q-network algorithm includes constructing a main Q-network and a target Q-network, which are used for estimating the action value function and providing a stable target value, respectively. The Q-value is trained through deep learning until convergence, and the current optimal number of blind retransmissions is selected using the trained Q-value function. In the example of this application, to achieve intelligent adaptive adjustment of the blind retransmission strategy, this application adopts the Dual Deep Q-Network (DDQN) algorithm to solve the aforementioned reinforcement learning problem. This method constructs two independent neural network structures, namely the main Q-network and the target Q-network. The main Q-network is responsible for evaluating the Q-value of each optional action (i.e., different blind retransmission times) in the current state in real time, while the target Q-network provides a stable reference value to guide the gradient update of the main Q-network, thereby effectively solving the problem of Q-value overestimation in the traditional DQN algorithm. During the training process, both the main Q-network and the target Q-network adopt multi-layer fully connected neural networks until the Q-value function gradually converges to a stable state. Finally, the trained Q-network can output the Q-value corresponding to each action based on the current communication state vector input. The agent selects the blind retransmission number with the largest Q-value as the execution strategy at the current moment, achieving the optimal balance between communication reliability and channel load in dynamic environments. This effectively improves the adaptability and response speed of the vehicle-to-everything (V2X) communication system in complex dynamic environments, ensuring the success rate of data transmission while preventing excessive channel load caused by excessive retransmissions. Overall, it achieves dynamic collaborative optimization of communication reliability and channel efficiency.

[0095] As a preferred example of this application, in step S5, the main Q-network and the target Q-network adopt a multi-layer fully connected neural network, and the current observed two-dimensional state vector is input. Output and retransmission count Nre p= 0, 1, 2, 3 — corresponding Q values, the main network parameter is θ, used for actual behavior selection and value update, the target network parameter is θ - This is used to construct a stable target Q value, and the Q value update formula is:

[0096]

[0097] Where α is the learning rate, β is the discount factor, and r t For in s t Take action a in the state t The reward obtained later is used to execute the strategy using an ε-greedy action selection method. This means that an action is randomly selected with a probability of ε to increase exploration, and the action with the highest current Q value is selected with a probability of 1-ε to ensure policy utilization. The training data comes from a local experience replay pool. After an action is taken, the environment will perceive the new state. And calculate the reward based on the reward function, then (s t a t r t s t+1 ) Store the samples in the local experience replay pool and use a small batch for training. t Let a be the state observed by the agent at time step t. t For the agent to select the number of blind retransmissions at time step t, r t To determine the action a taken by the agent at time step t. t The reward value is the feedback value from the environment, which is determined by the reward function taking into account communication reliability (such as effective propagation distance). ) and channel load (e.g., NCBR) t+1 )Calculations show that s t+1 For the new environmental state perceived by the agent at time step t+1, the TD error training objective is constructed by sampling the state-action-reward-next state quadruple, and the loss function is defined as the sum of squared TD errors:

[0098]

[0099] Where D represents the set of state-action pairs, y Q This represents the expected target Q value. The expression is as follows:

[0100] During training, the current Q-value is calculated using the main network parameters θ, and the target Q-value y is constructed using the target network parameters θ-. Q The TD error is used as the loss function for backpropagation to optimize the main network parameters θ. The target network copies parameters from the main network every certain training period to stabilize the target calculation. The entire training process involves continuous interaction, accumulation, and iteration, eventually causing the Q-value function to converge to the optimal policy estimate.

[0101] This application effectively improves the accuracy of Q-value estimation and the stability of policy convergence by introducing a dual-Q network structure and a TD error-driven learning mechanism into blind retransmission optimization. The target network, as a fixed-delay estimator, avoids the Q-value fluctuation problem caused by the main network during rapid updates, significantly reducing the risk of the policy getting trapped in local optima or overfitting. At the same time, the ε-greedy policy dynamically balances the policy exploration in the early training stage with the stable utilization in the later stage. Combined with the experience replay mechanism, it improves the utilization efficiency of training samples and breaks the temporal correlation between samples. The loss function constructs the optimization direction with the temporal difference (TD) error, making the training process more convergent and numerically stable. In the highly dynamic vehicle network environment, it can quickly complete policy updates and output the optimal number of blind retransmissions in real time, improving communication reliability and avoiding channel congestion. It is suitable for vehicle-to-vehicle communication blind retransmission policy optimization in various service types and traffic scenarios.

[0102] This application proposes a blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication. By constructing a reinforcement learning model with the effective communication range (ETR) and the net channel busy rate (NCBR) as core inputs, it introduces high-precision perception and statistical mechanisms to achieve dynamic modeling of the communication state. Combined with a dual-Q network structure constructed by a multi-layer neural network and a TD error-driven strategy learning framework, it can achieve precise adaptive adjustment of the number of blind retransmissions in complex and ever-changing vehicle-to-everything (V2X) communication environments. Compared with traditional strategies that use fixed retransmission counts or static rule judgments, this application's solution significantly improves the policy's environmental adaptability and real-time response capability. Through blind retransmission strategy optimization based on deep reinforcement learning, it dynamically adjusts the number of blind retransmissions in V2V communication, optimizes the effective communication distance and the net channel busy rate, and solves the channel overload problem in high-density scenarios.

[0103] The embodiments of this application have been described above with reference to the accompanying drawings. Unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other. This application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for optimizing blind retransmission strategies based on deep reinforcement learning in V2V communication, characterized in that, include: S1: The observation vehicle calculates different observation distances based on the received perception messages. r The packet reception rate (PRR) at the observation node is any vehicle participating in V2V communication. The packet reception rate (PRR) is the ratio of the number of data packets successfully received by the observation node to the total number of data packets sent by the neighboring nodes. S2: Based on the different observation distances obtained r The packet reception rate (PRR) at the location is used to estimate the effective communication range (ETR), where the effective communication range (ETR) is the communication distance that satisfies the preset packet reception rate (PRR) threshold. S3: Calculate the net busy rate (NCBR) of the channel under the blind retransmission mechanism based on the signal-to-noise ratio (SINR) of the received signal. S4: Construct a blind retransmission strategy optimization problem based on the effective communication range (ETR) and the net busy rate (NCBR) of the channel, and map the optimization problem into a deep reinforcement learning problem. Design the corresponding state space, action space and reward function. The state space includes at least the effective communication range (ETR) and the net busy rate (NCBR) of the channel. The action space includes different blind retransmission times. The reward function considers both communication reliability and channel load. In step S4, the state space is defined by the current effective propagation distance. A two-dimensional state vector composed of the channel net busy rate (NCBR). , t This indicates that at the current moment, the action space is the set of blind retransmission counts. The reward function is in, and These are weighting coefficients, satisfying... + =1, , , The maximum possible net channel busy rate; S5: Using vehicle nodes as reinforcement learning agents, the optimization problem is solved by employing a dual-deep Q-network algorithm to dynamically obtain the optimal number of blind retransmissions, which is then applied to subsequent data broadcasting.

2. The blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication according to claim 1, characterized in that, The packet reception rate (PRR) is the rate at which the observation node performs during the observation period. The number of data packets successfully received from neighboring nodes within the observation period and the number of neighboring nodes during the observation period. The ratio of the total number of data packets sent within the period, where: （ i =1,……, N ） in, N The number of neighboring nodes. For the observation node during the observation period Successfully received neighbor node i The number of data packets sent. For nodes i The total number of data packets sent during the observation period. This represents the number of packets sent per second, measured in Hertz (Hz).

3. The blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication according to claim 1, characterized in that, The neighboring nodes are determined based on their distance from the observed node. d Grouping observation nodes into different observation distance intervals, each observation node calculates its belonging to its respective distance interval. L The average packet reception rate of the neighboring nodes is taken as the PRR at the observation distance r, denoted as PRR(r), and the calculation formula is as follows: , This is the index of the observed nodes within the observation distance interval.

4. The blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication according to claim 1, characterized in that, In step S2, based on the different observation distances calculated in step S1 r The packet reception rate (PRR) at the location was obtained using a polynomial fitting method. r The curve is then used to estimate the effective communication range (ETR), where the effective communication range (ETR) is defined as the packet reception rate (PRR) at different observation distances being greater than a preset threshold for packet reception rate. The maximum communication distance, i.e. ； in, To meet Effective propagation distance, packet reception rate preset threshold Given the packet reception rate (PRR) threshold.

5. The method for optimizing blind retransmission strategy based on deep reinforcement learning in V2V communication according to claim 1, characterized in that, In step S3, the Net Channel Busyness Rate (NCBR) is defined as the signal-to-noise ratio (SINR) of the first data packet in a series of retransmitted data packets exceeding a SINR threshold. Time allocation: ， in, The first data packet in a series of retransmitted packets to have a signal-to-noise ratio (SINR) exceeding the SINR threshold. The time taken by the data packets, Observation time, each node every Update the Net Busy Rate (NCBR) of the channel.

6. The blind retransmission strategy optimization method based on deep reinforcement learning in V2V communication according to claim 5, characterized in that, In step S3, for the first j Signal-to-noise ratio (SINR) of each data packet signal Establish the following model: ， in, This refers to the signal receiving power. Noise This represents the average noise power. Representative at the j The average interference power received by the receiving node during the reception of each data packet.

7. The method for optimizing blind retransmission strategy based on deep reinforcement learning in V2V communication according to claim 1, characterized in that, The average interference power The weighted average of the interference signal strengths within the data packet reception period is obtained, and the calculation formula is shown below: ， in, For data packet transmission time, For nodes i , j The number of time periods divided during transmission, in the th n Time period The internal interference power always remains constant. For the time period Internal to receiving node j The set of signals that cause interference. For nodes k The proportion of frequency overlap between the emitted interference signal and the useful signal.

8. The method for optimizing blind retransmission strategy based on deep reinforcement learning in V2V communication according to claim 1, characterized in that, In step S5, the dual-deep Q-network algorithm includes constructing a main Q-network and a target Q-network, which are used to estimate the action value function and provide the target value, respectively. The Q-value is trained through deep learning until convergence, and the current optimal number of blind retransmissions is selected using the trained Q-value function.

9. The method for optimizing blind retransmission strategy based on deep reinforcement learning in V2V communication according to claim 1, characterized in that, In step S5, the main Q-network and the target Q-network adopt a multi-layer fully connected neural network, and the input is a two-dimensional state vector. Output four Q values ​​corresponding to the number of blind retransmissions. The main Q network parameters are: The target Q-network parameters are The Q-value update formula is: in For learning rate, As a discount factor, In order to be in Take action in a state The reward obtained later This represents the Q-value predicted by the online Q-network. This represents the Q-value predicted by the target Q-network, used during training. ε - A greedy strategy is used for action selection, and the loss function is optimized based on temporal difference (TD) error: ， Where D represents the set of state-action pairs, This represents the expected target Q value, which is: .

Images