Wireless ad hoc network vehicle-mounted base station

Abstract

The invention discloses a wireless ad hoc network vehicle-mounted base station, which comprises one or more receiving modules used for receiving a data packet from a vehicle; the one or more sending modules are used for sending data packets to the vehicle; the processing unit is used for optimizing the contention window through the following steps that S1, a Q value table is initialized, and the Q value table comprises the contention window and corresponding actions; judging whether in a training mode, if so, updating a random exploration ratio and a learning rate, and if not, selecting an action according to the current random exploration ratio and learning rate to adjust a competition window; s2, sending data to the vehicle according to the adjusted contention window; s3, calculating a reward updating Q value table according to the data packet returned by the vehicle; and S4, repeating the steps S1 to S3 until the Q value table is converged to obtain the optimal contention window. According to the method, the contention window value can be dynamically adjusted to adapt to different network environment changes, and the utilization rate of channel resources and the network performance are improved.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and more particularly to a wireless ad hoc network vehicle-mounted base station. Background Art

[0002] With the rapid development of intelligent transportation systems (ITS), vehicle-mounted ad hoc networks (VANETs), as an emerging wireless communication technology, are playing an increasingly important role in vehicle-to-everything (V2X) applications. VANETs allow real-time information exchange between vehicles and between vehicles and roadside infrastructure, thus enabling a variety of applications such as safe driving, traffic management, and infotainment.

[0003] However, existing VANET technology faces some challenges, one of which is the channel sharing problem. Since VANET networks are usually composed of a large number of moving vehicles and have no fixed infrastructure, channel resources are limited and packet conflicts are prone to occur, which reduces network performance and affects application effects.

[0004] The IEEE 802.11p protocol is one of the main technical standards for VANET, which defines a channel access control mechanism based on carrier sense multiple access / collision avoidance (CSMA / CA). The CSMA / CA mechanism controls the timing of vehicle data packet transmission by setting contention window (CW) parameters to avoid conflicts. However, the setting of CW parameters in the IEEE 802.11p protocol is usually static and cannot be adaptively adjusted according to changes in the network environment, resulting in low channel utilization and poor fairness, making it difficult to meet the needs of different types of applications. Summary of the invention

[0005] In view of this, in order to solve the above technical problems, the present invention provides a wireless ad hoc network vehicle-mounted base station.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] A wireless ad hoc network vehicle-mounted base station, comprising:

[0008] One or more receiving modules, for receiving data packets from the vehicle;

[0009] One or more sending modules, used to send data packets to the vehicle;

[0010] A processing unit is configured to optimize the contention window by:

[0011] S1. Initialize a Q value table, wherein the Q value table includes a contention window and a corresponding action;

[0012] Determine whether it is in training mode. If so, update the random exploration ratio and learning rate. If not, select an action based on the current random exploration ratio and learning rate to adjust the competition window.

[0013] S2. Send data to the vehicle according to the adjusted contention window;

[0014] S3. Calculate the reward and update the Q value table according to the data packet returned by the vehicle;

[0015] S4. Repeat steps S1 to S3 until the Q value table converges to obtain the optimal contention window.

[0016] Furthermore, the random exploration ratio and learning rate are updated according to the following formula;

[0017] ε t+1 =ε t -Δε×t×R n

[0018] a t+1 =a t -Δa×t×R n

[0019] In the formula, ε represents the random exploration ratio, a represents the learning rate, t represents the number of iterations, Δε and Δa represent the preset attenuation, and R n Represents the average reward in the last n steps; when updating parameters, the present invention considers R n , which can find a balance between exploration and exploitation, thereby converging to the optimal strategy faster.

[0020] Further, an action is selected according to the current random exploration ratio and learning rate to adjust the competition window; the steps include:

[0021] Randomly generate a random number m between 0 and 1,

[0022] If m≤ε, from the interval ((CW t -1) / 2,CW t ,CW t *2-1) randomly selects an action as a t+1 ;

[0023] If m ≥ ε, take the optimal action as a t+1 ;

[0024] According to action a t+1 Adjust the contention window.

[0025] Further, according to the data packet returned by the vehicle, the reward is calculated to update the Q value table, the steps include:

[0026] Verify whether the vehicle return data packet ID is consistent with the original sent data packet, and the return time is less than p seconds;

[0027] If not, set the reward to -1;

[0028] If yes, the reward is calculated considering the popularity of the CW value.

[0029] Furthermore, the rewards are calculated taking into account the popularity of the CW value, and the steps include:

[0030] Initialize the CW set, listen to the CW in the data frame of the neighboring sending module, and add it to the CW set.

[0031] When the CW set is full, the oldest CW value is removed;

[0032] Count the frequencies of different CW values ​​in the CW set and calculate the reward based on the frequencies.

[0033] Furthermore, rewards are calculated based on frequency, including:

[0034] Dividing the CW frequencies into different levels in advance, and setting corresponding first reward factors for different levels;

[0035] The reward is calculated according to the CW frequency and the corresponding first reward factor.

[0036] Further, a second reward factor is set according to the CW value, specifically: a higher reward is assigned to a smaller CW value, and a lower reward is assigned to a larger CW value, so as to reduce the total delay of data packet transmission; and

[0037] The first reward factor and the second reward factor are combined to calculate a reward.

[0038] Opportunistic monitoring is used to monitor adjacent sending modules.

[0039] The monitoring content includes the data packets sent by the adjacent sending module and the data packets returned by the receiving module.

[0040] When a transmission failure is detected, the uplink MU-MIMO is used to retransmit the erroneous subframe to the receiving module.

[0041] Further, the receiving module estimates the channel matrix according to the retransmitted erroneous subframe and the known signal of the transmitting module;

[0042] The received signal is linearly transformed using the inverse matrix of the channel matrix to eliminate the influence of the channel matrix and obtain the original data;

[0043] The original data is then decoded and demodulated to obtain the final received data.

[0044] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a wireless self-organizing network vehicle-mounted base station, which can dynamically adjust the contention window (CW) value to adapt to different network environment changes by adopting the Q-Learning algorithm, thereby improving the utilization of channel resources and network performance, and realizing adaptive learning and management of the network.

[0045] In addition, by considering the popularity of CW values ​​to calculate rewards, fairness in the network can be promoted. At the same time, by setting a second reward factor and allocating different rewards according to the size of the CW value, the total delay of data packet transmission can be reduced. By adjusting the weight of the reward function, the present invention can ultimately achieve a balance between the two sub-goals of fairness and low latency.

[0046] Another beneficial effect of the present application is that, by utilizing opportunistic monitoring and uplink MU-MIMO technology, the present invention can improve transmission efficiency, and in particular, can quickly retransmit erroneous subframes when transmission fails, thereby improving network reliability.

[0047] And by estimating the channel matrix and using its inverse matrix for linear transformation, the robustness of the system is improved, so that the network can maintain stable performance in a dynamically changing environment.

[0048] In general, the present invention effectively solves the channel sharing problem and network performance problem existing in the existing VANET technology through various technical means, thereby improving the overall performance and application effect of the VANET network and providing new possibilities for the application of vehicle-mounted wireless networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0050] Figure 1 A schematic diagram of the structure of a wireless ad hoc network vehicle-mounted base station provided by the present invention;

[0051] Figure 2 This is a flow chart of data processing in the processing unit provided by the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The embodiment of the present invention discloses a wireless ad hoc network vehicle-mounted base station, the structure of which is as follows: Figure 1 As shown, including:

[0054] One or more receiving modules, for receiving data packets from the vehicle;

[0055] One or more sending modules, used to send data packets to the vehicle;

[0056] A processing unit is configured to optimize a contention window by performing the following steps.

[0057] Furthermore, in order to realize adaptive adjustment of CW parameters according to changes in the network environment, the present application provides a contention window optimization method, wherein the specific optimization steps are as follows: Figure 2 As shown, including:

[0058] S1. Initialize a Q value table, wherein the Q value table includes a contention window and a corresponding action;

[0059] Determine whether it is in training mode. If so, update the random exploration ratio and learning rate. If not, select an action based on the current random exploration ratio and learning rate to adjust the competition window.

[0060] S2. Send data to the vehicle according to the adjusted contention window;

[0061] S3. Calculate the reward and update the Q value table according to the data packet returned by the vehicle;

[0062] S4. Repeat steps S1 to S3 until the Q value table converges to obtain the optimal contention window.

[0063] In this embodiment, initializing the Q value table is the first step of the present invention. The Q value table is a data structure for storing contention windows (CWs) and corresponding actions. When initializing the Q value table, all Q values ​​are set to initial values, such as 0. The initialization of the Q value table is to record and update the Q value of each contention window and action for subsequent Q value learning and optimization.

[0064] Furthermore, after initializing the Q value table, it is necessary to determine whether it is in training mode. The training mode is the process of learning the competition window adjustment strategy.

[0065] In this embodiment, if it is in training mode, it is necessary to update the random exploration ratio ε and the learning rate α; both are key parameters in the Q-Learning algorithm, used to balance exploration and exploitation; if it is not in training mode, an action is selected based on the current random exploration ratio and learning rate to adjust the competition window. In a preferred embodiment, the random exploration ratio and learning rate are updated according to the following formula:

[0066] ε t+1 =ε t -Δε×t×R n

[0067] a t+1 =a t -Δa×t×R n

[0068] In the formula, ε represents the random exploration ratio, a represents the learning rate, t represents the number of iterations, Δε and Δa represent the preset attenuation, and R n Represents the average reward in the last n steps.

[0069] Specifically, after performing a series of actions, a series of rewards will be obtained. By calculating the average of these rewards, we can understand whether the current behavior leads to positive rewards, thereby affecting the learning strategy. Among them, n is a pre-set parameter; in one embodiment, if n is 10, and the rewards obtained are 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 respectively, then R n is 5.5 (calculated as the average of the rewards for 10 steps).

[0070] During training, if R n If R is high, then you may be more likely to take actions similar to your previous actions, since this will usually result in positive rewards. n If R is low, it will choose to explore new actions in the hope of finding better rewards. n is an important feedback signal that helps adjust its behavior strategy in Q-Learning to optimize its long-term reward.

[0071] The method of adjusting the learning rate and exploration ratio based on the performance in the most recent iterations in the present invention can dynamically respond to learning progress and environmental changes, thereby achieving a better balance between exploring new strategies and using known strategies, and converging to the optimal strategy faster; at the same time, it avoids overconfidence in a certain strategy, even if the strategy performs well for a period of time. If the agent performs poorly in several consecutive iterations, it will increase the values ​​of ε and α accordingly to encourage the agent to explore new strategies, thereby avoiding falling into the local optimal solution.

[0072] In one embodiment, an action is selected according to the current random exploration ratio and learning rate to adjust the competition window; the steps include:

[0073] Randomly generate a random number m between 0 and 1,

[0074] If m≤ε, from the interval ((CW t -1) / 2,CW t ,CW t *2-1) randomly selects an action as a t+1 ;

[0075] If m ≥ ε, take the optimal action as a t+1 ;

[0076] According to action a t+1 Adjust the contention window.

[0077] This balance between exploitation and exploration guarantees convergence and produces good performance.

[0078] Then, data is sent to the vehicle according to the adjusted contention window; and data packets returned by the vehicle are received;

[0079] Furthermore, the present application calculates the reward and updates the Q value table according to the returned data packet, and the specific steps include:

[0080] Verify that the vehicle's returned data packet ID is consistent with the original sent data packet, and that the return time is less than p seconds; preferably, p is 0.1 seconds;

[0081] If not, set the reward to -1;

[0082] If yes, the reward is calculated considering the popularity of the CW value, the steps include:

[0083] Initialize the CW set, listen to the CW in the data frame of the neighboring sending module, and add it to the CW set.

[0084] When the CW set is full, the oldest CW value is removed;

[0085] Count the frequencies of different CW values ​​in the CW set and calculate the reward based on the frequencies.

[0086] The present application monitors and analyzes the CW information of data packets sent by adjacent sending modules, and assists in selecting the optimal CW for transmission to reduce collisions and improve network efficiency.

[0087] To further optimize the above technical solution, rewards are calculated based on frequency, including:

[0088] Dividing the CW frequencies into different levels in advance, and setting corresponding first reward factors for different levels;

[0089] The reward is calculated according to the CW frequency and the corresponding first reward factor.

[0090] In one embodiment, a second reward factor is further set according to the CW value, specifically: a higher reward is assigned to a smaller CW value, and a lower reward is assigned to a larger CW value, so as to reduce the total delay of data packet transmission; and

[0091] The first reward factor and the second reward factor are combined to calculate a reward.

[0092] To further optimize the above technical solution, opportunistic monitoring is used to monitor the adjacent sending modules. Opportunistic monitoring is a flexible monitoring method. The device monitors the channel only when it needs to transmit or receive data, thereby improving spectrum utilization.

[0093] In the present application, when monitoring the CW of the data packet sent by the adjacent sending module, the data packet data and the data packet returned by the receiving module are monitored at the same time, so that when a transmission failure is monitored, the uplink MU-MIMO is used to retransmit the erroneous subframe to the receiving module.

[0094] Uplink MU-MIMO means that multiple transmitters send data to one receiver at the same time, that is, by using spatial multiplexing technology to improve transmission efficiency without increasing bandwidth; while ensuring data integrity.

[0095] Furthermore, in order to effectively suppress multipath interference and improve signal quality, the receiving module estimates the channel matrix based on the retransmitted erroneous subframe and the known signal of the transmitting module;

[0096] The received signal is linearly transformed using the inverse matrix of the channel matrix to eliminate the influence of the channel matrix and obtain the original data;

[0097] The original data is then decoded and demodulated to obtain the final received data.

[0098] The present invention effectively solves the channel sharing problem and network performance problem existing in the existing VANET technology through the above-mentioned multiple technical means, thereby improving the overall performance and application effect of the VANET network.

[0099] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0100] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A wireless ad hoc network vehicle-mounted base station, characterized in that: include: One or more receiving modules, for receiving data packets from the vehicle; One or more sending modules, used to send data packets to the vehicle; A processing unit is configured to optimize the contention window by: S1. Initialize a Q value table, wherein the Q value table includes a contention window and a corresponding action; Determine whether it is in training mode. If so, update the random exploration ratio and learning rate. If not, select an action based on the current random exploration ratio and learning rate to adjust the competition window. S2. Send data to the vehicle according to the adjusted contention window; S3. Calculate the reward and update the Q value table according to the data packet returned by the vehicle; S4. Repeat steps S1 to S3 until the Q value table converges to obtain the optimal contention window.

2. The wireless ad hoc network vehicle-mounted base station according to claim 1, characterized in that: Update the random exploration ratio and learning rate according to the following formula; e t+1 =e t -D×t×R n a t+1 =a t -Δa×t×R n In the formula, ε represents the random exploration ratio, a represents the learning rate, t represents the number of iterations, Δε and Δa represent the preset attenuation, and R n Represents the average reward in the last n steps.

3. The wireless ad hoc network vehicle-mounted base station according to claim 1, characterized in that: Select actions based on the current random exploration ratio and learning rate to adjust the competition window; the steps include: Randomly generate a random number m between 0 and 1, If m≤ε, from the interval ((CW t -1) / 2,CW t ,CW t *2-1) randomly selects an action as a t+1 ; If m ≥ ε, take the optimal action as a t+1 ; According to action a t+1 Adjust the contention window.

4. The wireless ad hoc network vehicle-mounted base station according to claim 1 or 2, characterized in that: According to the data packet returned by the vehicle, the reward is calculated and the Q value table is updated. The steps include: Verify whether the vehicle return data packet ID is consistent with the original sent data packet, and the return time is less than p seconds; If not, set the reward to -1; If yes, the reward is calculated considering the popularity of the CW value.

5. The wireless ad hoc network vehicle-mounted base station according to claim 4, characterized in that: Considering the popularity of CW value, the reward is calculated. The steps include: Initialize the CW set, listen to the CW in the data frame of the neighboring sending module, and add it to the CW set. When the CW set is full, the oldest CW value is removed; Count the frequencies of different CW values ​​in the CW set and calculate the reward based on the frequencies.

6. The wireless ad hoc network vehicle-mounted base station according to claim 5, characterized in that: Rewards are calculated based on frequency, including: Dividing the CW frequencies into different levels in advance, and setting corresponding first reward factors for different levels; The reward is calculated according to the CW frequency and the corresponding first reward factor.

7. The wireless ad hoc network vehicle-mounted base station according to claim 6, characterized in that: Setting a second reward factor according to the CW value, specifically: assigning a higher reward to a smaller CW value and a lower reward to a larger CW value, so as to reduce the total delay of data packet transmission; as well as The first reward factor and the second reward factor are combined to calculate a reward.

8. The wireless ad hoc network vehicle-mounted base station according to claim 5, characterized in that: Opportunistic monitoring is used to monitor adjacent sending modules.

9. The wireless ad hoc network vehicle-mounted base station according to claim 8, characterized in that: The monitoring content includes the data packets sent by the adjacent sending module and the data packets returned by the receiving module. When a transmission failure is detected, the uplink MU-MIMO is used to retransmit the erroneous subframe to the receiving module.

10. The wireless ad hoc network vehicle-mounted base station according to claim 9, characterized in that: The receiving module estimates the channel matrix based on the retransmitted erroneous subframe and the known signal of the transmitting module; The received signal is linearly transformed using the inverse matrix of the channel matrix to eliminate the influence of the channel matrix and obtain the original data; The original data is then decoded and demodulated to obtain the final received data.

Images