A time window-based random estimation self-learning neighbor discovery method

Abstract

The application discloses a time window-based random estimation self-learning neighbor discovery method. The application belongs to the field of communication network neighborhood and mainly solves the problems of neighbor node discovery difficulty and long convergence time in a directional self-organizing network. In the method, network nodes in the neighbor discovery problem are modeled as learning automata, then a random estimator and a time window-based optimization method are used, so that the automata can adapt to the changing environment and adjust the selection probability of each sector in real time, thereby accelerating the neighbor discovery speed in the wireless self-organizing network scene using a directional antenna. MATLAB simulation results prove that the algorithm can accelerate the discovery process and is robust to complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of communication networks, and in particular, it is a method for intelligent neighbor discovery suitable for narrow-beam directional antennas. Background Technology

[0002] In recent years, with the advancement of directional antenna technology and wireless communication technology, directional antennas have been widely used in many wireless networks. The use of directional antennas concentrates energy on the target receiver, providing not only longer transmission distances and better spatial multiplexing, but also increased network capacity. Narrow-beam antennas, with their high main lobe gain and low sidelobe gain, offer better performance, but also present challenges, especially in self-organizing networks. To form a self-configuring network, neighbor discovery is the first step in network formation, enabling wireless nodes to establish a one-hop link with other nodes. In directional antennas using half-duplex transmit / receive modes, two nodes can only communicate with each other if their beams are aligned and they operate in complementary receive and transmit modes.

[0003] Over the past few decades, researchers have proposed numerous directional neighbor discovery methods to address the challenges encountered in neighbor discovery within wireless ad hoc networks using directional antennas. These methods include scan-based and random-based approaches. In scan-based methods, nodes periodically probe all sectors by switching beams in a predefined sequence. This method requires some prior information about the neighbor distribution to ensure the efficiency of the scan sequence, and the scan sequence remains largely unchanged during neighbor discovery. In probabilistic methods, nodes randomly select a sector for neighbor discovery in each time slot. Without prior information, probabilistic methods offer greater robustness and flexibility in ad hoc networks, but their discovery efficiency is lower.

[0004] Recent research has proposed reinforcement learning-based methods to address the challenges of directional neighbor discovery. These methods largely utilize reinforcement learning algorithms to learn from the discovery process, leveraging environmental feedback to adjust sector selection probabilities and accelerate the process. Directional neighbor discovery methods based on learning automata directly adjust probabilities based on experience. However, these methods only consider learning in a fixed environment, while directional neighbor discovery is a non-fixed learning environment, also known as a non-stationary learning environment. Therefore, these methods cannot effectively solve the neighbor discovery problem.

[0005] This invention considers a scenario where wireless ad hoc networks using narrow-beam directional antennas perform neighbor discovery without prior information. To discover potential neighbors quickly and reliably, a reinforcement learning method adaptable to the non-stationary learning environment of neighbor discovery is required, thereby improving system robustness and reducing the time overhead of neighbor discovery. Summary of the Invention

[0006] The purpose of this invention is to address the scenario of neighbor discovery in wireless ad hoc networks using narrow-beam directional antennas without prior information. This invention proposes a time-window-based random estimation self-learning neighbor discovery method, which can quickly and effectively discover neighbor nodes without prior information. The problem is described as follows: each wireless node is considered an independent learning automaton, and the combination of all nodes is considered the environment for node interaction. Each node selects an action to interact with the environment according to a probability vector in each time slot, receives feedback from the environment, and adjusts the probability vector using the feedback through an algorithm. Then, it randomly selects an action based on the new probability vector, repeating the above process. To achieve this purpose, the steps employed in this invention are as follows:

[0007] Step 1: Model each node in the network as an independent learning automaton. The set of all nodes is considered the environment in which the learning automaton interacts. Each learning automaton mainly consists of three parts: the first part is experience acquisition, composed of a deterministic estimator and a window function, used to handle environmental feedback. The second part is stochastic estimation, composed of a stochastic estimator, used to more accurately estimate environmental features. The third part is probability vector adjustment, which adjusts the probability vector based on the results of the first two parts, enabling the learning automaton to better adapt to non-fixed learning environments and better solve the neighbor discovery problem.

[0008] Step 2: For each node modeled as a learning automaton, without any prior information, each node sets its initial action probability vector P. i (t) is set to an equal value, i.e., 1 / r, where r represents the total number of sectors of the node. Next, taking any node i as the object, the algorithm implementation process is explained in detail. In the first time slot after neighbor discovery begins, node i randomly selects sector k to send or receive data packets according to the initial probability vector. At the end of this time slot, the node, based on environmental feedback B... i Construct a deterministic estimation vector D using a time window function. i The estimated value of the k-th sector is defined as the number of times R is rewarded within the window time. k and the number of times C was selected k The ratio of .

[0009] Step 3: Add a normal distribution parameter N to construct a random estimation vector U i That is, the random estimation vector U i The value is the deterministic estimation vector D. i The value is the sum of the normal distribution parameter N. The normal distribution parameter N is a standardized random number with a mean of 0 and a variance proportional to the time elapsed since the last selected sector k, denoted as αm. k(t), where α is an internal parameter of the automaton, representing the rate at which the random estimate deviates from the deterministic estimate. The maximum value of the variance of the normal distribution parameter N is constrained to α. max To avoid extreme cases, at the end of each time slot, the probability of the sector with the highest estimate in the random estimator is increased. Assuming that sector j has the highest reward value at this time, the probability vector is updated by the following formula:

[0010]

[0011] Here, λ represents the learning rate, and its value is between 0 and 1.

[0012] Step 4: During the neighbor discovery process, some sectors of node i may have only a very few neighbors or no neighbors at all. Therefore, a criterion is introduced: if sector j has never discovered a neighbor within a time window of β times its normal value, its probability is adjusted to an extremely low value, as shown in the following formula:

[0013]

[0014] After the adjustment is complete, node i will randomly select a new action in the next time slot using the probability vector updated in steps 3 and 4, and interact with the environment again. Steps 2, 3, and 4 will then be repeated.

[0015] The proposed time-window-based stochastic estimation self-learning neighbor discovery method has been implemented in MATLAB, and simulation results verify its superiority. The simulation experiment assumes that nodes are randomly deployed in a 1000m*1000m square region. (Appendix) Figure 1 This means that each node in the network is modeled as a learning automaton. A learning automaton consists of three parts: experience acquisition, random estimation, and probability adjustment. (Appendix) Figure 2 This is a schematic diagram of the time window function. (Attached) Figure 3 This compares the neighbor discovery speed of WSLA with other algorithms under the conditions of a beamwidth of 15° and a total of 210 nodes. (Appendix) Figure 4 This section compares the convergence time of WSLA with other algorithms under different numbers of nodes. Figure 5 This comparison shows the convergence time of WSLA compared to other algorithms under different sector widths. It demonstrates the superiority of the proposed time-window-based stochastic estimation self-learning neighbor discovery method. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the system model;

[0017] Figure 2 This is a schematic diagram of the time window function;

[0018] Figure 3This is a simulation result diagram of the speed detected by the neighbor;

[0019] Figure 4 This is a simulation result graph showing the convergence time for different numbers of nodes;

[0020] Figure 5 The graph shows the convergence time simulation results for different sector widths. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings and examples.

[0022] In the following description, this specification will refer to the intelligent neighbor discovery method proposed in this invention for narrow-beam directional antennas—a time window-based stochastic learning automaton (WSLA)—as WSLA. WLSA first sets the following system parameters:

[0023] 1. Motion space: A i An action indicates that a node randomly selects to send or receive data packets using a sector in a specific time slot.

[0024] 2. Probability vector: P i (t), the probability vector represents the probability that a node selects a particular sector in a given time slot t. It is important to note that the sum of the probabilities of all sectors selected by each node is always 1.

[0025] 3. Environmental Feedback: B i After a node performs an action and interacts with the environment, it receives feedback from the environment, which can be in three states: collision detected, successful discovery of a new neighbor, or idle. The first two environmental feedback values ​​are defined as 1, and the idle state value is defined as 0.

[0026] 4. Determine the estimated vector: D i (t), determine the value d of the estimated vector. k The number of times R is rewarded within the window time w k and the number of times C was selected k The ratio is determined by this.

[0027] 5. Randomly estimated vector: U i (t), where the random estimation vector contains the learning automaton's estimation of environmental features.

[0028] 6. Normal distribution parameters: N, including the learning machine's internal parameter α, representing the rate at which the random estimate deviates from the deterministic estimate; parameter m k (t) represents the time elapsed since the last sector k was selected; parameter α max , representing the maximum value of the variance.

[0029] 7. Learning algorithm: T, in each time slot, the agent updates the probability vector through the learning algorithm.

[0030] Based on the above conditions, the WSLA proposed in this invention has been implemented in MATLAB, and the results demonstrate the effectiveness of the method. The specific implementation steps of WSLA are as follows:

[0031] Step 1: Model each node in the network as an independent learning automaton. The set of all nodes is considered the environment in which the learning automaton interacts. Each learning automaton mainly consists of three parts: the first part is experience acquisition, composed of a deterministic estimator and a window function, used to handle environmental feedback. The second part is stochastic estimation, composed of a stochastic estimator, used to more accurately estimate environmental features. The third part is probability vector adjustment, which adjusts the probability vector based on the results of the first two parts, enabling the learning automaton to better adapt to non-fixed learning environments and better solve the neighbor discovery problem.

[0032] Step 2: For each node modeled as a learning automaton, without any prior information, each node sets its initial action probability vector P. i (t) is set to an equal value, i.e., 1 / r, where r represents the total number of sectors of the node. Next, taking any node i as the object, the algorithm implementation process is explained in detail. In the first time slot after neighbor discovery begins, node i randomly selects sector k to send or receive data packets according to the initial probability vector. At the end of this time slot, the node, based on environmental feedback B... i Construct a deterministic estimation vector D using a time window function. i The estimated value d of the kth sector k R is defined as the number of times sector k is rewarded within the window time. k and the number of times C was selected k The ratio. Specifically:

[0033] C k (t+1)=C k +1 (3)

[0034] R k (t+1)=R k (t)+B k (t) (4)

[0035]

[0036] Step 3: Add a normal distribution parameter N to construct a random estimation vector U. i That is, the random estimate u k The value is the deterministic estimation vector d. k The sum of the value and the normal distribution parameter N. Expressed as:

[0037]

[0038] The normal distribution parameter N is a standardized random number with a mean of 0 and a variance σ. k It is proportional to the time elapsed since the last sector k was selected, denoted as αm. k (t), where α is an internal parameter of the automaton, representing the rate at which the random estimate deviates from the deterministic estimate. The maximum value of the variance of the normal distribution parameter N is constrained to α. max This is to avoid some extreme situations from occurring.

[0039] At the end of each time slot, the probability of the sector with the highest estimated value in the random estimator is increased. Assuming that sector j has the highest reward value at this time, the probability vector is updated using formula (1):

[0040] Step 4: During the neighbor discovery process, some sectors of node i may have only a very small number of neighbors or no neighbors at all. Therefore, a criterion is introduced: if sector j has never discovered a neighbor within a time window of β times the time window, its probability is adjusted to an extremely low value, as shown in formula (2):

[0041] After the adjustment is complete, node i will randomly select a new action in the next time slot using the probability vector updated in steps 3 and 4, and interact with the environment again. Steps 2, 3, and 4 will then be repeated.

[0042] The contents not described in detail in this application are existing technologies known to those skilled in the art.

Claims

1. A time-window-based stochastic estimation self-learning neighbor discovery method, comprising the following steps: Step 1: Model each node in the network as an independent learning automaton. The set of all nodes is considered as the environment for the interaction of the learning automata. Each learning automaton consists of three parts: The first part is experience acquisition, which consists of a deterministic estimator and a window function, used to handle environmental feedback; The second part is stochastic estimation, which consists of stochastic estimators and is used to estimate environmental characteristics more accurately. The third part is probability vector adjustment, which adjusts the probability vector based on the results of the first two parts, so that the learning automaton can better adapt to non-fixed learning environments and better solve the neighbor discovery problem. Step 2: For each node modeled as a learning automaton, without any prior information, each node sets its initial action probability vector P. i (t) is set to an equal value, i.e., 1 / r, where r represents the total number of sectors of the node; taking any node i as the research object, when node i is in the first time slot after neighbor discovery, it randomly selects sector k to send or receive data packets according to the initial probability vector. At the end of the time slot, the node determines the data packet based on the environmental feedback B. i Construct a deterministic estimation vector D using a time window function. i The estimated value of the k-th sector is defined as the number of times R is rewarded within the window time. k and the number of times C was selected k The ratios are as follows: C k (t+1)=C k +1 (1) R k (t+1)=R k (t)+B k (t) (2) Step 3: Add normal distribution parameter N to construct random estimation vector U i That is, the random estimation vector U i The value is the deterministic estimation vector D. i The sum of the value and the normal distribution parameter N is expressed as: The normal distribution parameter N is a standardized random number with a mean of 0 and a variance proportional to the time elapsed since the last sector selection k, denoted as αm. k (t), where a is an internal parameter of the automaton, representing the rate at which the random estimate deviates from the deterministic estimate; the maximum value of the variance of the normal distribution parameter N is limited to a. max To avoid extreme cases, after each time slot ends, the probability of the sector with the largest estimate in the random estimator is increased; assuming that the reward value of sector j is the highest at this time, the probability vector is updated by the following formula: Here, λ represents the learning rate, and its value is between 0 and 1. Step 4: During the neighbor discovery process, some sectors of node i may have only a very few neighbors or no neighbors at all. Therefore, a criterion is introduced: if sector j has never discovered a neighbor within a time window of β times its normal value, its probability is adjusted to an extremely low value, as shown in the following formula: After the adjustment is completed, node i will randomly select a new action in the next time slot based on the probability vector updated in steps 3 and 4, interact with the environment again, and repeat steps 2, 3, and 4.

Images