A connected unmanned aerial vehicle cross-layer perception cell handover method
By combining cross-layer state space and multi-head self-attention mechanism, the problem of unstable switching of connected UAVs in highly dynamic environments is solved, achieving efficient communication decision-making and service continuity, and improving the reliability of UAV communication.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FUDAN UNIVERSITY
- Filing Date
- 2026-05-13
- Publication Date
- 2026-07-31
AI Technical Summary
Existing cell handover methods for connected drones struggle to effectively handle variable-length neighbor cell information in highly dynamic environments, leading to unstable handover decisions, frequent handovers, and communication interruptions.
By constructing a cross-layer state space, introducing a multi-head self-attention mechanism for feature aggregation, and combining a multi-step experience replay mechanism and deep reinforcement learning, the switching decision model is optimized to achieve joint perception and adaptive decision-making of physical layer channel quality and transport layer service status.
It improves the accuracy and stability of handover decisions, reduces frequent handovers and link interruptions, balances communication quality and service continuity, and provides reliable communication assurance.
Smart Images
Figure CN122496876A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of wireless communication technology for connected drones, and specifically relates to a method for cross-layer sensing cell handover for connected drones. Background Technology
[0002] In recent years, with the rapid development of the low-altitude economy and drone technology, connected drones have been increasingly widely used in logistics delivery, power line inspection, and emergency communications. However, in complex low-altitude environments, the reliable transmission and mobility management of connected drones still face severe challenges.
[0003] Existing terrestrial cellular network architectures are primarily designed for ground users. The downtilt antenna configuration causes drones to receive fragmented sidelobe signals at high altitudes. Furthermore, due to the lack of obstructions at high altitudes, drones can easily establish line-of-sight links with multiple base stations simultaneously, leading to severe signal fluctuations and interference. Secondly, drones possess high-speed, three-dimensional mobility, resulting in a much higher handover frequency than ground users. This easily triggers frequent "ping-pong effects" at the edges of base stations. Unnecessary handovers or handover failures directly lead to communication interruptions and high latency, seriously threatening the reliability of control command issuance and high-bandwidth data transmission.
[0004] Cell handover algorithms have evolved from static threshold configuration to dynamic adaptive optimization. Threshold comparison-based algorithms are the most basic triggering mechanisms, such as the A3 event defined by 3GPP. Reference [1] reduced the handover failure rate to a certain extent by adjusting static parameters such as offset and trigger time. However, its performance is highly dependent on the parameters manually set by the network administrator and lacks the ability to adapt to dynamic network environments. To overcome the defects of static thresholds, reinforcement learning techniques with environmental awareness and sequence decision-making capabilities have become an important research direction. Reference [2] proposed a handover strategy called SMART and designed a scheme based on the upper confidence bound (UCB) algorithm for different scenarios. Although the number of handovers was reduced, the strategies learned by reinforcement learning methods such as UCB were difficult to generalize to states that had not been experienced, and the sampling efficiency was limited in large-scale scenarios such as ultra-dense networks. With the emergence of deep reinforcement learning (DRL), Reference [3] proposed a scheme based on the asynchronous advantage actor-commentator (A3C) framework, which uses long short-term memory networks to extract temporal features. Although this scheme outperforms traditional algorithms in terms of throughput, its complex two-layer learning architecture faces significant computational overhead in UAV scenarios with extremely high real-time requirements. Reference [4] further proposes a method based on deep deterministic policy gradient (DDPG), which sets the state space as the ratio of signal received power to signal interference noise, aiming to regulate the boundary value of continuous handover. Although this method outperforms traditional strategies in reducing unnecessary handovers, existing DRL schemes generally focus on the static mapping of physical layer indicators, failing to fully consider the severe impact on the throughput of the transport layer during handover, and their neural network architecture lacks flexibility in handling the dynamically changing number of neighboring base stations. Summary of the Invention
[0005] The purpose of this invention is to propose a cross-layer sensing cell handover method for networked unmanned aerial vehicles (UAVs) to overcome the problems of insufficient processing capability for variable-length neighbor cell information, inadequate utilization of cross-layer information, and poor decision-making stability in existing cell handover methods in highly dynamic UAV scenarios.
[0006] The proposed cross-layer sensing cell handover method for networked unmanned aerial vehicles (UAVs) takes the physical layer channel quality and transport layer service status during UAV communication as joint sensing objects. It constructs a cross-layer state space, introduces a multi-head self-attention mechanism to achieve adaptive feature aggregation of variable-length neighbor cell information, and combines a multi-step experience replay mechanism to establish a stable and efficient handover decision model. This achieves an adaptive trade-off between throughput and handover overhead in highly dynamic channel environments. The specific steps are as follows:
[0007] Step S1: Real-time acquisition of physical layer signal (signal-to-interference-to-noise ratio of SINR) measurements of the serving cell and neighboring cells of the UAV at the current moment; simultaneous acquisition of real-time service throughput indicators of the transmission layer; and extraction of connection duration and historical handover frequency characteristics with the current cell to jointly construct a multi-dimensional cross-layer state space. ;
[0008] Step S2, Feature Reconstruction and Dimension Alignment of Multi-Head Self-Attention Mechanism: The state space vector constructed in Step S1 is... The input is fed into the "attention-enhanced deep neural network", which sequentially passes through the feature encoding layer, the multi-head self-attention layer and the action value evaluation layer to complete the feature representation learning and action value estimation.
[0009] Step S3: Based on the action value estimation results output in S2, determine the switching action using a preset action selection strategy. It then executes the corresponding cell handover operation; monitors the environmental evolution after the action is executed, and constructs an instantaneous reward function by combining the SINR gain, throughput fluctuation, and handover frequency constraints before and after the handover. ;
[0010] Step S4, the model is optimized using a multi-step experience replay mechanism and a joint cost function. Specifically, the experience replay mechanism [5] is used to store the state transition quadruples generated by the UAV during environmental interaction. , , , The algorithm constructs training samples containing long-term time dependencies based on an n-step reward mechanism. Under the deep reinforcement learning framework, it uses the action value function to approximate the network to calculate the action value estimate of the current state-action pair, and constructs the target value based on the cumulative reward of the n steps and the long-term state value estimate. By measuring the deviation between the action value estimate and the target value, it constructs the mean squared error loss function, and updates the network parameters in reverse based on the gradient descent algorithm to achieve iterative optimization and convergence of the model parameters.
[0011] Step S5: After the DRL model training is completed, lock the neural network parameters and cancel the random exploration mechanism to enter the deterministic decision-making mode; deploy the DRL model on the UAV terminal to perform online inference, and use the trained model to infer the real-time cross-layer state under the preset test trajectory. By statistically analyzing and outputting key performance indicators such as handover success rate, average throughput and ping-pong handover rate, the robustness evaluation and verification of the cell handover method in dynamic scenarios is completed.
[0012] Furthermore:
[0013] In step S1, the construction of a multi-dimensional cross-layer state space The specific process is as follows: When the instruction is triggered in each decision cycle, the system synchronously acquires the uplink signal-to-noise ratio signals of the current serving base station and neighboring base stations. And combined with the real-time service throughput feedback from the transport layer Simultaneously, record the connection duration of the drone in the current cell. And the past Cumulative number of switches within a time window After normalizing the above parameters, they are concatenated into an input vector. :
[0014]
[0015] in, The number of base stations that can be detected within the drone's communication coverage area is a variable parameter that changes dynamically with the drone's geographical location.
[0016] In step S2, the feature representation learning and action value estimation specifically include the following sub-steps:
[0017] Step S2-1, convert the state space vector constructed in step S1 into a vector. Expanded along the base station dimension, the feature matrix is used, where each row corresponds to a combination of the local sensing features and the global service status of a candidate base station. For heterogeneous indicators with different dimensions such as physical layer signal-to-noise ratio, transmission layer throughput, and handover frequency, the input features are first normalized, and then projected onto a unified high-dimensional feature space through a feature encoding layer composed of linear mapping to obtain an aligned feature representation matrix.
[0018] Step S2-2: Input the feature representation matrix obtained in step S2-1 into the multi-head self-attention layer, and use the learnable query weight matrix. Key weight matrix Sum weight matrix Query vector Q, key vector K, and value vector V are generated respectively. By calculating the dot product similarity between Q and K and combining it with scaling factors and Softmax normalization, the attention weight distribution among candidate base stations is obtained, thereby characterizing the contribution of different neighboring base stations to the current decision.
[0019] Step S2-3 involves using the attention weights obtained in step S2-2 to perform a weighted summation of the value vector V, generating an enhanced feature representation that aggregates global spatial correlation information. This process maps the variable-length input, which dynamically changes with the number of neighboring regions, to a fixed-dimensional feature vector, thereby eliminating the impact of inconsistent input dimensions on subsequent decisions.
[0020] Step S2-4: Input the fixed-dimensional feature vector obtained in step S2-3 into the action value evaluation layer. This layer consists of multiple fully connected networks, which abstract the features layer by layer through nonlinear transformations, and finally output the current state. The vector of action value estimates corresponding to each discrete action below In this invention, the action space is defined as a discrete set encompassing all available base station indices within the UAV's communication coverage area, and its mathematical expression is:
[0021]
[0022] in, This represents a discrete index sequence of N available base stations within the drone's communication coverage area.
[0023] In step S3, the process of determining the switching action using an action selection strategy and constructing an instantaneous reward function specifically includes the following sub-steps:
[0024] Step S3-1: The system obtains the action value estimate vector output in step S2-4. Then, using The strategy is used for decision mapping. The system presets an exploration factor. ,by The probability search is used to find the maximum value in the vector of estimated action values, in order to The probability is randomly selected within the action space, and the final output is the target action. .
[0025] Step S3-2, perform the target action determined in step S3-1. Switch to a toggle decision. If the target action... If the index is the same as the current serving cell, it is determined that no handover is required, and the drone continues to maintain communication with the current cell; if the determination result is to handover to the target neighboring cell base station, the drone terminal initiates a handover request to the current serving cell and executes the signaling process of disconnecting the current connection and establishing a new bearer with the target base station.
[0026] Step S3-3: Within a preset observation window after the action is executed, the gain of the physical layer signal quality and the fluctuation data of the transport layer service throughput before and after the handover are sensed in real time, and these observations are substituted into the instantaneous reward function formula to calculate the reward value. This reward value serves as a direct feedback signal to evaluate the merits of this handover decision and is used for model optimization in the subsequent step S4. The instantaneous reward function... The calculation formula is:
[0027]
[0028] In the formula, , These are the preset weighting coefficients; This represents the signal quality of the target base station after the action is executed, and is used to ensure physical layer connectivity. This represents the transport layer throughput, used to sense the stability of service transmission. This is a pulse penalty term for a single handover, which takes the value when a base station handover action occurs. Otherwise ; To represent the cumulative number of switching times over the past W time windows, a linear penalty is applied to frequent switching behavior to suppress the ping-pong effect.
[0029] Step S4, the optimization model using the multi-step experience replay mechanism and joint cost function, includes the following sub-steps:
[0030] Step S4-1: During the interaction between the UAV and the environment, construct the state transition quadruple ( The data is then stored in the experience pool; during the training phase, a data sequence of n consecutive time steps starting from time t is extracted from the experience pool. Training samples containing long-term temporal dependencies are constructed based on the sequences.
[0031] Step S4-2: Randomly extract a batch of sample data of a predetermined size from the aforementioned experience pool. Under the deep reinforcement learning framework, a dual-network structure (training network and target network) is used for value evaluation. The training network is used to calculate the action value estimate of the current state-action pair, and the target network is used to calculate the target value. The specific process is as follows: The current state in the sample... Input the training network to obtain the action value estimate at the current time step. Simultaneously, based on the instantaneous reward values recorded in the extracted sample sequence for n consecutive cycles... The cumulative reward is calculated. (in (This is a preset discount factor). And the target network is used to analyze the long-term state. Conduct a valuation to obtain a future value estimate. Construct the action at that moment. Actual feedback reference value under the current strategy :
[0032]
[0033] Step S4-3: Utilize the estimated value of the action calculated in step S4-2. Compared with actual feedback reference value The mean squared error between the training and target networks is used to construct a mean squared error loss function, and the gradient descent algorithm is used to update the parameters of the training network in reverse. Simultaneously, at a preset update period, the parameters of the training network are synchronized to the target network to maintain the stability of the target value estimation. Through this iterative optimization process, the model parameters gradually converge, thereby learning the optimal switching strategy.
[0034] Step S5 involves deploying the DRL model on a drone terminal to perform online inference after training, specifically including the following sub-steps:
[0035] Step S5-1: The system acquires the real-time cross-layer status. Then, input it into the already trained DRL model. At this point, let the exploration factor... This means eliminating the random exploration mechanism and directly selecting the index with the highest score from the output action value estimate vector as the target action. .
[0036] Step S5-2, according to The system compares the current base station index with the current base station index to determine whether a handover is needed. If a handover is needed, the drone terminal sends a handover request to the current base station, disconnects from the current base station, and establishes a connection with the target base station.
[0037] Step S5-3: After the UAV connects to the target base station, the system collects and calculates key performance indicators, including handover frequency and average throughput.
[0038] In summary, this invention utilizes a measurement system to acquire the physical layer signal quality and real-time transport layer traffic throughput of the UAV to construct a joint state space. Then, a multi-head self-attention mechanism is used to extract features and model the associations of variable-length neighboring cell base station states, achieving feature reconstruction of the action value of each candidate base station. Based on a deep reinforcement learning (DRL) architecture, a deep neural network is used to evaluate and decide on each candidate handover action to obtain the optimal handover action and improve the accuracy of the value function estimation. Finally, a multi-step cumulative reward method is used to map the reward signal constructed from signal gain and throughput fluctuations, along with network training errors, into a joint cost function. The agent parameters are iteratively updated using a gradient descent algorithm, and the optimal handover decision information is output in real time. This invention solves the problem of frequent handovers and link interruptions caused by drastic channel fluctuations in highly dynamic, three-dimensional, and complex environments, effectively balancing communication quality and service continuity, and providing a reliable communication guarantee solution for networked UAV systems.
[0039] The main technical features and functional advantages of this invention are as follows:
[0040] (1) By introducing a cross-layer state modeling mechanism, joint perception of information from the physical layer and the transport layer is realized, which improves the accuracy of handover decisions;
[0041] (2) By using a multi-head self-attention mechanism to model the variable-length neighbor information, the problem of inconsistent input dimensions caused by the dynamic change of the number of neighbor cells is effectively solved;
[0042] (3) By introducing a dual-network structure and a multi-step reward mechanism, the stability of model training is effectively improved;
[0043] (4) Through the design of a multi-objective reward function, an adaptive balance between system throughput and switching overhead is achieved. Attached Figure Description
[0044] Figure 1 This is a flowchart of the cell handover method for cross-layer perception of networked drones according to the present invention.
[0045] Figure 2 This is a flowchart illustrating the working mechanism of the attention-enhanced deep reinforcement learning model according to an embodiment of the present invention.
[0046] Figure 3 This is a comparison chart of the average throughput and cumulative number of handovers between the present invention embodiment and existing handover algorithms. Detailed Implementation
[0047] The present invention will be further described below with reference to the embodiments and accompanying drawings.
[0048] Figure 1 This is an overall flowchart of the cross-layer sensing cell handover method for networked unmanned aerial vehicles proposed in this invention; Figure 2 This is a flowchart illustrating the internal processing of the attention-enhanced deep reinforcement learning model in this invention, used for... Figure 1 The steps described are explained in detail below. These include the following steps:
[0049] Step S1: First, the system receives cross-layer data input from the UAV's onboard communication module. At the physical layer, it acquires the real-time signal-to-interference-noise ratio (SINR) measurement between the serving cell and neighboring cells. Simultaneously, at the transport layer, it collects service throughput metrics and extracts the connection duration between the UAV and the current cell, as well as handover frequency characteristics from historical trajectories. These heterogeneous parameters collectively construct a multi-dimensional cross-layer state space. .
[0050] Step S2, create the state space The input is fed into a pre-defined attention-enhanced deep neural network, passing sequentially through a feature encoding layer, a multi-head self-attention layer, and an action value evaluation layer to complete feature representation learning and action value estimation. This process includes the following sub-steps:
[0051] Step S2-1, convert the state space vector Expanded along the base station dimension, it is a feature matrix X each of the lines This represents the combination of the local channel characteristics and the global service state of the i-th base station, i.e.:
[0052]
[0053] Then, for heterogeneous features with different dimensions, normalization is first performed, and then the features are projected onto a unified high-dimensional feature space through a feature encoding layer constructed by linear mapping to obtain the feature representation matrix:
[0054]
[0055] in, is the weight matrix of the feature embedding layer, which maps the original input features from a low-dimensional space to a unified high-dimensional feature space.
[0056] Step S2-2: Input the feature representation matrix H into the multi-head self-attention layer. Generate the query vector, key vector, and value vector using three learnable weight matrices:
[0057]
[0058] in, , and This represents the trainable weight matrix in the attention mechanism; the query vector Q and key vector K are used to characterize the correlation between features from different base stations; and the value vector V carries the feature information to be aggregated. The above matrix... , , and It originates from the initialization of the neural network and is continuously updated during the training process.
[0059] Then use and The dot product operation measures the spatial distribution correlation and handover gain potential among different candidate base stations, and generates an attention weight matrix after Softmax normalization:
[0060]
[0061] in, This is the attention weight matrix, used to characterize the correlation strength between candidate base stations; The dimension of the key vector is used to scale the dot product results to stabilize the training process; the Softmax operation normalizes the data row by row so that the sum of the weights in each row is 1.
[0062] Step S2-3, adjust the attention weight matrix. AND value vector Weighted summation is performed to output an enhanced feature vector that aggregates global spatial information. This operation uses weighted aggregation to compress variable-length neighbor cell features into a single fixed-dimensional feature, thereby eliminating the interference of dynamic changes in the number of base stations on the subsequent deep reinforcement learning input layer and solving the problem that traditional fully connected networks cannot handle dynamic topologies.
[0063] Step S2-4: Input the fixed-dimensional enhanced feature vector output from step S2-3 into the action value evaluation layer. This layer consists of multiple fully connected networks, which abstract the features layer by layer through nonlinear mapping, and finally output the current state. The vector of estimated action values corresponding to each discrete action index below .
[0064] Step S3, obtaining the motion value estimate vector Next, the system enters the decision-making and reward evaluation phase. First, an ε-greedy strategy is used to select the target action. Subsequently, a cell handover decision is executed based on the selected action: if the target base station is the same as the current serving base station, the connection is maintained; otherwise, the handover procedure is executed. Within the observation window after the action is executed, changes in environmental conditions are monitored, and an instantaneous reward function is constructed by combining the SINR gain, throughput changes, and handover overhead before and after the handover. .
[0065] Step S4: The system enters the model parameter optimization stage. During the interaction between the UAV and the environment, a state transition quadruple is constructed ( The data is stored in the experience pool; during the training phase, a data sequence of n consecutive time steps starting from time t is extracted from the experience pool. Training samples containing long-term temporal dependencies are constructed based on the sequences.
[0066] Subsequently, within the deep reinforcement learning framework, a dual-network structure is employed for value evaluation, where the training network is used for action selection and the target network is used for value evaluation. The system calculates the cumulative reward over n steps based on the sample sequence and constructs the target value by combining the value estimate of the long-term state. A mean squared loss function is constructed by measuring the difference between the action value estimate and the target value, and the network parameters are updated using the gradient descent algorithm, thereby achieving iterative optimization and convergence of the model.
[0067] Step S5: After model training converges, the system locks all mean parameters of the neural network and cancels the random exploration mechanism to enter deterministic inference mode. Under the preset test trajectory, the trained model is used to perform online inference on real-time cross-layer states. By statistically analyzing the handover frequency and average throughput, the robustness of the cell handover method in dynamic scenarios is evaluated.
[0068] For a comparison of the number of handovers and throughput with other similar algorithms under the same environment and trajectory, see [link to relevant documentation]. Figure 3 .
[0069] Example: The experimental environment was built on the MATLAB R2025b simulation platform, and a high-fidelity wireless communication scenario was constructed using the 5G Toolbox. The hardware platform used an Intel Core i7 processor, and the software platform was based on a Windows 10 system and an integrated Python deep learning interface.
[0070] 1. Simulation Scenario and Parameter Initialization
[0071] This embodiment uses MATLAB 5G Toolbox scripts to build a high-fidelity wireless communication simulation environment. A heterogeneous network topology with 5 base stations is configured within the geographical area. The drone's flight altitude is fixed at 120 meters, and its flight speed is set at 20 m / s.
[0072] During the model training phase, the system generates multiple sets of random motion trajectories, enabling the UAV to perform irregular flights within the overlapping coverage areas of various base stations. This aims to enhance the generalization learning capability of the DRL model for unknown channel environments by traversing complex scenarios.
[0073] During the model verification phase, this embodiment pre-sets 10 fixed test trajectories. These trajectories are set in the MATLAB simulation environment to cover a variety of representative motion paths, such as straight-line crossing, S-shaped detour, and tangential flight at the edge of the base station.
[0074] In the specific verification process, these 10 preset trajectories were run respectively, and the entire process data of the UAV completing the flight mission under each trajectory was recorded. Subsequently, these 10 sets of experimental data were summarized, and the arithmetic mean of the average throughput and the cumulative number of handovers was calculated. This average value was used as the final performance indicator to evaluate the robustness of the cell handover method.
[0075] 2. Implementation details of the DRL algorithm
[0076] In this embodiment, the DRL algorithm is implemented using a Double Deep Q Network (DDQN) architecture. The DDQN comprises a training network and a target network with consistent structures, which together constitute an approximation model of the action value function. The training network is used for action selection, and the target network is used for target value evaluation.
[0077] Specifically, the action value function approximation model employs an attention-enhanced deep neural network structure, which sequentially includes a state encoding layer, a multi-head self-attention layer, and an action value evaluation layer. The state encoding layer consists of a linear embedding layer and two fully connected layers, each containing 128 hidden units, with LayerNormalization introduced between layers to improve training stability. The multi-head self-attention layer uses a 4-head structure with a feature dimension of 128, used for correlation modeling and weighted aggregation of variable-length neighborhood features. The action value evaluation layer consists of a fully connected network, ultimately outputting the Q-value estimate for each candidate action.
[0078] During the training process, the following methods were adopted: The strategy achieves a dynamic balance between exploration and utilization. Training samples are constructed based on an n-step reward mechanism, where the step size n is set to 3 to account for both short-term channel quality changes and long-term service performance evolution. Model parameters are updated by minimizing the mean squared error loss between the predicted Q-value and the target value. The optimizer uses the Adam algorithm with a learning rate of 0.0005 and a discount factor γ of 0.95. Simultaneously, to improve training stability, a gradient pruning mechanism is introduced to limit the gradient norm to within 1.0.
[0079] In addition, the parameter synchronization period between the training network and the target network is set to be once every 500 training steps.
[0080] The DRL algorithm in S3 uses the same "deep neural network" in the Double Deep Q Network (DDQN) architecture as the "deep neural network layer" in the attention-enhanced deep reinforcement learning model in step S2.
[0081] The overall workflow of the "attention-enhanced deep neural network" is consistent with the process in step S2. Specifically, the input state is first processed by a state encoding layer for feature alignment and dimension mapping to obtain a unified high-dimensional feature representation H. Subsequently, H is input to a multi-head self-attention layer, where variable-length neighbor features are mapped to a fixed-dimensional global feature representation through correlation modeling and weighted aggregation operations. Finally, this feature is input to an action value evaluation layer, which outputs the Q-value estimate corresponding to each candidate base station.
[0082] Step S3 selects actions based on the Q-value estimation results described above. The DDQN architecture includes a training network and a target network with consistent structures, both of which are constructed using the aforementioned "attention-enhanced deep neural network" structure.
[0083] 3. Performance Comparison and Metric Validation of Multiple Algorithms
[0084] During the evaluation phase, this embodiment introduces three typical algorithms as a comparison group: the 3GPP A3 event algorithm based on a fixed threshold, the UCB algorithm based on online learning, and the DDPG algorithm based on continuous parameter optimization. Figure 3 shows the performance comparison between the method of this invention and the comparison algorithms on the two core indicators of "system average throughput" and "cumulative handover count" under the same test trajectory. The data shows that the method of this invention significantly reduces unnecessary handover frequency while ensuring high throughput.
[0085] The above embodiments demonstrate that, due to the adoption of the cross-layer perception DDQN architecture, this invention can achieve more accurate environmental perception and more robust decision output than traditional algorithms when facing complex low-altitude interference and high-speed motion scenarios in 5G environments. This not only provides network-connected drones with more accurate network status perception and connection decision capabilities, but also significantly improves the end-to-end energy efficiency of the communication system by suppressing the ping-pong effect.
[0086] The above embodiments are only used to illustrate specific implementations of the present invention, and the present invention is not limited to the above embodiments.
[0087] References
[0088] [1] LOBINGER A, STEFANSKI S, JANSEN T, et al. Coordinating handoverparameter optimization and load balancing in lte self-optimizing networks[C] / / 2011 IEEE 73rd Vehicular Technology Conference (VTC Spring). IEEE, 2011:1-5.
[0089] [2] SUN Y, FENG G, QIN S, et al. The smart handoff policy formillimeter wave heterogeneous cellular networks[J]. IEEE Transactions onMobile Computing, 2017, 17(6): 1456-1468.
[0090] [3] WANG Z, LI L, XU Y, et al. Handover control in wireless systemsvia asynchronous multiuser deep reinforcement learning[J]. IEEE Internet ofThings Jour nal, 2018, 5(6): 4296-4307.
[0091] [4] KWONG C F, SHI C, LIU Q, et al. Autonomous handover parameteroptimisation for 5g cellular networks using deep deterministic policygradient[J]. Expert Systems with Applications, 2024, 246: 122871.[5] Mnih V,Badia A P, Mirza M, et al. Asynchronous methods for deep reinforcementlearning[C] / / International Conference on Machine Learning. PmLR, 2016: 1928-1937。
Claims
1. A method for cross-layer sensing cell handover of a networked unmanned aerial vehicle (UAV), characterized in that, Taking the physical layer channel quality and transport layer service status in UAV communication as joint sensing objects, this paper constructs a cross-layer state space, introduces a multi-head self-attention mechanism to achieve adaptive feature aggregation of variable-length neighbor cell information, and combines a multi-step experience replay mechanism to establish a stable and efficient handover decision model, thereby achieving an adaptive trade-off between throughput and handover overhead in a highly dynamic channel environment. The specific steps are as follows: Step S1, real-time acquisition of the physical layer signal measurement value of the service cell and the adjacent cell of the current time of the unmanned aerial vehicle, signal to interference and noise ratio SINR, synchronous acquisition of the real-time service throughput index of the transmission layer, and extraction of the connection duration and historical handover frequency characteristics of the current cell, to jointly construct a multi-dimensional cross-layer state space Step S2, constructing a state space vector for step S1 The input is input into the attention-enhanced deep neural network, sequentially passes through the feature encoding layer, the multi-head self-attention layer and the action value evaluation layer, and completes the feature representation learning and the action value estimation. Step S3: Based on the action value estimation results output in S2, determine the switching action using a preset action selection strategy. It then executes the corresponding cell handover operation; monitors the environmental evolution after the action is executed, and constructs an instantaneous reward function by combining the SINR gain, throughput fluctuation, and handover frequency constraints before and after the handover. ; Step S4: Optimize the model using a multi-step experience replay mechanism and a joint cost function; specifically, use the experience replay mechanism to store the state transition quadruples generated by the UAV during environmental interaction (…). , , , ), and construct training samples containing long-term temporal dependencies based on an n-step reward mechanism; Within the deep reinforcement learning framework, the action value function is used to approximate the network to calculate the action value estimate of the current state-action pair, and the target value is constructed based on the cumulative reward of the n steps and the long-term state value estimate. By measuring the deviation between the action value estimate and the target value, a mean squared error loss function is constructed, and the network parameters are updated in reverse based on the gradient descent algorithm to achieve iterative optimization and convergence of the model parameters. Step S5: After the DRL model training is completed, lock the neural network parameters and cancel the random exploration mechanism to enter the deterministic decision-making mode; The DRL model is deployed on a drone terminal to perform online inference. Under a preset test trajectory, the trained model is used to infer the real-time cross-layer state. By statistically analyzing and outputting performance indicators such as handover success rate, average throughput, and ping-pong handover rate, the robustness evaluation and verification of the cell handover method in dynamic scenarios are completed.
2. The method for cross-layer sensing cell handover of a networked unmanned aerial vehicle according to claim 1, characterized in that, In step S1, the construction of a multi-dimensional cross-layer state space The specific process is as follows: When the instruction is triggered in each decision cycle, the uplink signal-to-noise ratio (SNR) signals of the current serving base station and neighboring base stations are acquired synchronously. And combined with the real-time service throughput feedback from the transport layer Simultaneously, record the connection duration of the drone in the current cell. And the past Cumulative number of switches within a time window After normalizing the above parameters, they are concatenated into an input vector. : in, The number of base stations that can be detected within the communication coverage area of the drone is a variable parameter that changes dynamically with the geographical location of the drone.
3. The method for cross-layer sensing cell handover of a networked unmanned aerial vehicle according to claim 2, characterized in that, In step S2, the feature representation learning and action value estimation specifically include the following sub-steps: Step S2-1, convert the state space vector constructed in step S1 into a vector. Expanded into a feature matrix along the base station dimension, each row corresponds to a combination of the local sensing features and the global service status of a candidate base station. For heterogeneous indicators with different dimensions such as physical layer signal-to-noise ratio, transmission layer throughput and handover frequency, the input features are first normalized and then projected onto a unified high-dimensional feature space through a feature encoding layer composed of linear mapping to obtain an aligned feature representation matrix. Step S2-2: Input the feature representation matrix obtained in step S2-1 into the multi-head self-attention layer, and use the learnable query weight matrix. Key weight matrix Sum weight matrix Query vector Q, key vector K, and value vector V are generated respectively. By calculating the dot product similarity between Q and K and combining the scaling factor and Softmax normalization operation, the attention weight distribution among candidate base stations is obtained, thereby characterizing the contribution of different neighboring base stations to the current decision. Step S2-3: Use the attention weights obtained in step S2-2 to perform a weighted summation on the value vector V to generate an enhanced feature representation that aggregates global spatial correlation information; This process maps the variable-length input, which changes dynamically with the number of neighboring cells, into a fixed-dimensional feature vector, thereby eliminating the impact of inconsistent input dimensions on subsequent decisions. Step S2-4: Input the fixed-dimensional feature vector obtained in step S2-3 into the action value evaluation layer. This layer consists of multiple fully connected networks, which abstract the features layer by layer through nonlinear transformations, and finally output the current state. The vector of action value estimates corresponding to each discrete action below The action space is defined as a discrete set encompassing all available base station indices within the drone's communication coverage area, and its mathematical expression is: in, This represents a discrete index sequence of N available base stations within the drone's communication coverage area.
4. The method for cross-layer sensing cell handover of a networked unmanned aerial vehicle according to claim 3, characterized in that, In step S3, the process of determining the switching action using an action selection strategy and constructing an instantaneous reward function specifically includes the following sub-steps: Step S3-1: The system obtains the action value estimate vector output in step S2-4. Then, using The strategy is used for decision mapping; an exploration factor is preset. ,by The probability search is used to find the maximum value in the vector of estimated action values, in order to The probability is randomly selected within the action space, and the final output is the target action. ; Step S3-2, perform the target action determined in step S3-1. Convert to a switching decision; if the target action If the index is the same as the current serving cell, it is determined that no handover is required, and the drone continues to maintain communication with the current cell. If the decision is to switch to the target neighboring cell base station, the drone terminal will initiate a handover request to the current serving cell and execute the signaling process of disconnecting the current connection and establishing a new bearer with the target base station. Step S3-3: Within a preset observation window after the action is executed, the gain of the physical layer signal quality and the fluctuation data of the transport layer service throughput before and after the handover are sensed in real time, and these observations are substituted into the instantaneous reward function formula to calculate the reward value; this reward value serves as a direct feedback signal to evaluate the merits of this handover decision and is used for model optimization in the subsequent step S4; the instantaneous reward function... The calculation formula is: In the formula, , These are the preset weighting coefficients; This represents the signal quality of the target base station after the action is executed, and is used to ensure physical layer connectivity. This represents the transport layer throughput, used to sense the stability of service transmission. This is a pulse penalty term for a single handover, which takes the value when a base station handover action occurs. Otherwise ; To represent the cumulative number of switching times over the past W time windows, a linear penalty is applied to frequent switching behavior to suppress the ping-pong effect.
5. The method for cross-layer sensing cell handover of a networked unmanned aerial vehicle according to claim 4, characterized in that, Step S4, which involves optimizing the model using a multi-step experience replay mechanism and a joint cost function, includes the following sub-steps: Step S4-1: During the interaction between the UAV and the environment, construct the state transition quadruple ( The data is then stored in the experience pool; during the training phase, a data sequence of n consecutive time steps starting from time t is extracted from the experience pool. Training samples containing long-term temporal dependencies are constructed based on the sequences; Step S4-2: Randomly extract batches of sample data of a predetermined size from the aforementioned experience pool. Under the deep reinforcement learning framework, a dual-network structure is used for value evaluation. The training network is used to calculate the action value estimate of the current state-action pair, and the target network is used to calculate the target value. The specific process is as follows: The current state in the sample... Input the training network to obtain the action value estimate at the current time step. Simultaneously, based on the instantaneous reward values recorded in the extracted sample sequence for n consecutive cycles... The cumulative reward is calculated. ,in The discount factor is preset; and the target network is used to analyze the future state. Conduct a valuation to obtain a future value estimate. ; construct the action at that moment Actual feedback reference value under the current strategy : Step S4-3: Utilize the estimated value of the action calculated in step S4-2. Compared with actual feedback reference value The mean squared error between the training network parameters is used to construct a mean squared error loss function, and the gradient descent algorithm is used to update the training network parameters in reverse. At the same time, the training network parameters are synchronized to the target network at a preset update period to maintain the stability of the target value estimation. Through the above iterative optimization process, the model parameters gradually converge, thereby achieving the learning of the optimal switching strategy.
6. The method for cross-layer sensing cell handover of a networked unmanned aerial vehicle according to claim 5, characterized in that, In step S5, after the DRL model is trained, it is deployed on the drone terminal to perform online inference, which specifically includes the following sub-steps: Step S5-1: The system acquires the real-time cross-layer status. Then, input it into the already trained DRL model; at this point, let the exploration factor... This means eliminating the random exploration mechanism and directly selecting the index with the highest score from the output action value estimate vector as the target action. ; Step S5-2, according to Compare with the current base station index to determine whether a switch is needed. If a switch is needed, the drone terminal sends a switch request to the current base station, disconnects from the current base station, and establishes a connection with the target base station. Step S5-3: After the UAV connects to the target base station, key performance indicators, including handover frequency and average throughput, are statistically analyzed and calculated.