Physical layer authentication enhanced federated learning method for unmanned aerial vehicles

Abstract

The present application relates to the technical field of federated learning, and proposes a physical layer authentication enhanced UAV federated learning method, which comprises: judging whether the identity authentication of the edge device is qualified through a communication model based on physical layer authentication enhancement, if qualified, the UAV receives the local model of the federated learning updated by the edge device, if not qualified, the UAV refuses to receive the local model, wherein the communication model is a binary false detection test model using signal-to-noise ratio and false negative rate; a UAV scheduling resource model is constructed based on the false negative rate, and the UAV scheduling resource model is solved through an LSTM-SDRL algorithm to generate a UAV scheduling strategy for determining safety performance, communication performance, energy saving performance and model accuracy performance. The present application identifies malicious devices with unqualified identity authentication through physical layer identification, and guarantees the safety, communication performance, energy saving performance, model accuracy performance, high privacy and low communication overhead of federated learning.

Description

Technical Field

[0001] This invention relates to the field of federated learning technology, and more particularly to a federated learning method for unmanned aerial vehicles with enhanced physical layer authentication. Background Technology

[0002] To drive the next generation of networks toward ubiquitous intelligence and intrinsic security, utilizing drones (drones) to collect data from ground-based edge devices and extracting valuable information based on machine learning algorithms has become a promising approach. However, traditional centralized training methods for machine learning suffer from privacy risks and network traffic congestion, making it impractical for edge devices to transmit raw data to drones.

[0003] Federated learning paradigms allow edge devices to collaboratively train drone models by transmitting updated parameters instead of raw data, significantly improving privacy and communication efficiency. However, in open wireless environments, federated learning paradigms still present security challenges. For example, malicious nodes could impersonate legitimate nodes and exploit various attacks to intercept or compromise uploaded model parameters.

[0004] Currently, most research on ensuring the security of federated learning focuses on various cryptographic techniques, which typically have high computational costs and are unsuitable for widespread deployment in IoT devices. A key issue is how to perform efficient and reliable continuous authentication of resource-constrained edge devices during the training process of federated learning.

[0005] Existing federated learning security technologies include authentication protocols, homomorphic encryption, blockchain, and differential privacy. These commonly used technologies for ensuring federated learning security are typically only suitable for encryption and authentication between high-performance nodes and servers, and are not applicable to accessing massive numbers of edge devices in open wireless environments where communication and computing resources are limited.

[0006] In addition, existing physical layer authentication methods include fingerprinting: establishing a unique device fingerprint by analyzing the inherent radio frequency characteristics of communication devices, such as signal strength, phase and spectral characteristics; channel response: utilizing time-varying channel characteristics such as multipath propagation and delay spread; and phase characteristics: achieving authentication by analyzing signal phase changes.

[0007] While the aforementioned technologies offer different approaches to achieving physical layer authentication, they are not suitable for drone environments, especially for establishing high-capacity secure links between mobile drones and ground-edge devices.

[0008] Therefore, how drones can identify malicious devices through physical layer authentication technology to ensure the performance of federated learning is a technical problem that needs to be solved. Summary of the Invention

[0009] To address this, the present invention provides a physical layer authentication-enhanced UAV federated learning method that identifies malicious devices with ineligible authentication at the physical layer, thereby ensuring the security, communication performance, energy efficiency, model accuracy, high privacy, and low communication overhead of federated learning.

[0010] To achieve the above objectives, this invention proposes a physical layer authentication-enhanced UAV federated learning method, in which the UAV acts as a mobile server and performs federated learning with ground-based edge devices, including:

[0011] The edge device's authentication is determined by a communication model based on physical layer authentication enhancement. If the authentication is successful, the drone receives the local model updated by the edge device through federated learning. If the authentication is unsuccessful, the drone refuses to receive the local model. The communication model is a binary false detection model using signal-to-noise ratio and false negative rate.

[0012] Based on the false negative rate, a UAV orchestration resource model is constructed. The UAV orchestration resource model is solved using the LSTM-SDRL algorithm to generate a UAV orchestration strategy to determine the safety performance, communication performance, energy-saving performance, and model accuracy performance.

[0013] Furthermore, the local optimization problem of the communication model based on physical layer authentication enhancement aims to minimize the local loss function of the model variation between the global model and the local model of the edge device in the time slot. The local optimization problem includes a local variation loss term, a local loss function gradient term, and a global loss function gradient term.

[0014] The local variation loss term is a local loss function with the sum of the global model and model variations as variables;

[0015] The local loss function gradient term includes the gradient of the local loss function in the global model;

[0016] The gradient term of the global loss function includes the gradient of the global loss function in the global model.

[0017] Furthermore, the communication model performs a binary hypothesis test based on the difference in signal-to-noise ratio between adjacent global rounds of federated learning to determine whether the authentication of the edge device is qualified;

[0018] The probability density function of the binary hypothesis test is calculated based on the signal-to-noise ratio, Rayleigh fading, and channel deterministic variables.

[0019] Furthermore, the channel deterministic variables are determined based on the reference channel power gain, the distance between the UAV and the edge device, the transmit power of the edge device, and the noise power at the UAV.

[0020] The probability density function is derived from the communication relationship between the signal-to-noise ratio, the Rayleigh fading, and the channel deterministic variables.

[0021] Furthermore, the UAV orchestration resource model includes an objective function and constraints;

[0022] The objective function is to minimize the safety learning cost function based on UAV trajectory, federated learning participation variables, and latency and energy consumption variables.

[0023] The constraints include a comparison between the false negative rate and the security threat threshold, a minimum participating device constraint for federated learning based on the federated learning participation variables, and a drone motion constraint based on the drone trajectory.

[0024] Furthermore, the false negative rate is determined by a probability function based on Rayleigh fading rate, the difference in signal-to-noise ratio between malicious edge devices, and channel deterministic variables.

[0025] Furthermore, the LSTM-SDRL algorithm includes an LSTM sub-algorithm and an SDRL sub-algorithm;

[0026] The LSTM sub-algorithm is used to strengthen the expected cumulative reward network and the network for increasing the degree of violation of security constraints, thereby generating a strengthened expected cumulative reward network and a strengthened network for increasing the degree of violation of security constraints.

[0027] The SDRL sub-algorithm constructs a policy function based on the enhanced expected cumulative reward network and the enhanced security constraint violation degree network to solve for and generate the UAV orchestration policy.

[0028] Furthermore, the hidden state of the output layer of the LSTM sub-algorithm is connected to the SDRL sub-algorithm through a fully connected neural network.

[0029] Furthermore, the policy distribution function of the SDRL sub-algorithm is constructed based on the expected cumulative reward network, the expected degree of security constraint violation network, and the reward function.

[0030] Furthermore, the SDRL sub-algorithm constructs a reward function based on the negative penalty constraint of safe learning cost and false negative rate.

[0031] The above solution enables UAVs to perform efficient and reliable authentication of edge devices, achieve Pareto optimality in training accuracy, latency, and energy consumption, and provide four guarantees for UAVs to carry out edge device data mining tasks: security performance, communication performance, energy-saving performance, and model accuracy.

[0032] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0033] 1. By identifying malicious devices with ineligible authentication at the physical layer, the security, communication performance, energy efficiency, model accuracy, high privacy, and low communication overhead of federated learning are ensured.

[0034] 2. It enables UAVs to perform efficient and reliable authentication of edge devices, achieving Pareto optimality in training accuracy, latency, and energy consumption, and providing four guarantees for UAVs to carry out edge device data mining tasks: security performance, communication performance, energy-saving performance, and model accuracy. Attached Figure Description

[0035] Figure 1 This is a schematic diagram illustrating the general flow of the physical layer authentication-enhanced drone federated learning method according to an embodiment of the present invention.

[0036] Figure 2 This is a schematic diagram of the structure of the physical layer authentication-enhanced drone federated learning method according to an embodiment of the present invention;

[0037] Figure 3 This is a schematic diagram of the algorithm for the physical layer authentication-enhanced drone federated learning method according to an embodiment of the present invention. Detailed Implementation

[0038] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0039] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0040] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0041] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0042] like Figures 1 to 3 As shown, this invention provides a physical layer authentication-enhanced drone federated learning method that identifies malicious devices with unqualified identity authentication at the physical layer, thereby ensuring the security, communication performance, energy efficiency, model accuracy, high privacy, and low communication overhead of federated learning.

[0043] like Figures 1 to 3 As shown, this embodiment proposes a federated learning method for UAVs with enhanced physical layer authentication. The UAV acts as a mobile server and performs federated learning with ground-based edge devices, including:

[0044] The edge device's authentication is determined by a communication model based on physical layer authentication enhancement. If the authentication is successful, the drone receives the local model updated by the edge device through federated learning. If the authentication is unsuccessful, the drone refuses to receive the local model. The communication model is a binary false detection model using signal-to-noise ratio and false negative rate.

[0045] Based on the false negative rate, a UAV orchestration resource model is constructed. The UAV orchestration resource model is solved using the LSTM-SDRL algorithm to generate a UAV orchestration strategy to determine the safety performance, communication performance, energy-saving performance, and model accuracy performance.

[0046] It's important to note that federated learning is a distributed machine learning method that trains models on local devices and shares only the model parameters, not the raw data, enabling collaborative training among multiple parties while protecting data privacy and security. Physical layer authentication (PLA) is a technology that leverages the randomness of wireless channels and device hardware characteristics to achieve identity verification. By analyzing channel characteristics or physical layer information such as radio frequency fingerprints, it distinguishes between authorized users and unauthorized intruders, enhancing the security of communication systems. Compared to cryptographic-based authentication technologies, it has greater advantages on resource-constrained IoT devices.

[0047] Specifically, such as Figure 2 As shown, the UAV u remains in the urban environment, acting as a mobile parameter server for the federated learning (FLM) task, and is trained collaboratively with ground edge devices (EDs), denoted as... In the formula, Let i represent all qualified edge devices, i represent qualified edge devices that have passed authentication, and I represent the total number of qualified edge devices.

[0048] However, due to the openness of the wireless environment, malicious devices can masquerade as legitimate devices and disrupt tasks through various attacks, such as backdoor attacks and poisoning attacks. In the formula, J represents all malicious edge devices, and j represents an unauthorized edge device / malicious edge device with invalid authentication.

[0049] Specifically, the core steps of the federated learning include: defining a private dataset for each edge device i as follows: Where x and y represent the feature vector and label of each training data point, respectively, the edge device performs local training for federated learning by minimizing its local loss function, which is:

[0050]

[0051] In the formula, L i (w g ) represents the local loss function for the edge device, w g D represents the global model. i Represents a private dataset, x i,n ,y i,n Let l be the feature vector and label of the training data of the i-th edge device, and l() be the local loss function of the n-th private dataset of the i-th edge device.

[0052] Uploading the local model to the global model, the training of the global model can be represented as:

[0053]

[0054] In the formula, w g L represents the global model. i (w g D represents the local loss function of the i-th edge device. i Let D represent the private dataset, and let D represent the entire dataset.

[0055] in, This indicates that the entire dataset refers to the private dataset of all authenticated and qualified devices.

[0056] Furthermore, the local optimization problem of the communication model based on physical layer authentication enhancement aims to minimize the local loss function of the model variation between the global model and the local model of the edge device in the time slot. The local optimization problem includes a local variation loss term, a local loss function gradient term, and a global loss function gradient term. The local variation loss term is a local loss function with the global model and the sum of model variations as variables. The local loss function gradient term includes the gradient of the local loss function in the global model. The global loss function gradient term includes the gradient of the global loss function in the global model.

[0057] Specifically, the algorithm flow of this embodiment is roughly as follows: Step S1, Startup: Each qualified edge device reports its location and transmits the pilot signal to the drone for registration and system startup.

[0058] Step S2, Local Training and Model Upload: Each participating edge device trains a local model based on a private dataset and uploads it to address the following local optimization issues for physical layer authentication:

[0059]

[0060] In the formula, G i (w g,t ,h i,t ) is the authentication performance loss function, h i,t It is the change between the global model and the local model of the i-th edge device in the t-th time slot, which is used as a local adjustment term, w g,t Let L represent the global model for time slot t. i (w g,t +h i,t ) represents the local variation loss term, indicating the local loss function of the i-th edge device, which depends on w. g,t and h i,t ; It is the gradient term of the local loss function, representing the local loss function L. i In w g,t gradient; is the gradient term of the global loss function, representing the gradient of the global loss function L. Therefore, a physical layer authentication local optimization problem is defined, with the global model and the local model of federated learning as variables. In the formula, h i,t It is the change between the global and local models of the edge device in time slot t, w g,t Let L represent the global model for time slot t. i () represents the local loss function of the edge device, δ represents the update step size, and G i This represents local optimization issues for edge devices.

[0061] Step S3, UAV-Assisted Physical Layer Authentication (PLA): The UAV authenticates each updated parameter to distinguish unauthorized Byzantine nodes from qualified participating edge devices. Physical layer authentication is based on wireless characteristics, such as SNR differences. Detailed methods are described in the section on Physical Layer Authentication Based on Signal-to-Noise Ratio Difference.

[0062] Step S4, Edge Device Selection Enhanced with Physical Layer Authentication: The drone makes judgments based on physical layer authentication, refusing to accept models from nodes judged as malicious or from a few easily confused qualified nodes. The security device selection set is defined as follows: in K represents all qualified edge devices. t Represents all binary selection variables s i.t k i.t =1 indicates that device i is determined to be a qualified edge device in time slot t.

[0063] Step 5, Global Aggregation and Distribution: After collecting local models from all selected edge devices, the drone server merges them into a new global model iteration using a specific aggregation method. Subsequently, the drone broadcasts the updated global model w for round t. g,t And distribute it to all eligible edge devices to facilitate local model optimization in the subsequent t+1 round, which can be expressed as:

[0064]

[0065] In the formula, w g,t Let h represent the global model in round t. i,t It represents the change between the global and local models of the edge device in time slot t, where η represents the accuracy of the local model. This represents the local optimal gradient. The expression represents h. i,t The conditions for achieving local accuracy η.

[0066] Therefore, once the given global accuracy ∈ is reached, the entire process can terminate, that is:

[0067]

[0068] In the formula, L(w) g Let ) denote the global loss function in round t, and ∈ denote the global accuracy. This represents the globally optimal model.

[0069] In this embodiment, both the drone and the edge device are equipped with a single antenna, and can be easily extended to a multi-antenna configuration by applying physical layer authentication in MIMO communication.

[0070] Furthermore, the communication model performs a binary hypothesis test based on the difference in signal-to-noise ratio between adjacent global rounds of federated learning to determine whether the authentication of the edge device is qualified; wherein, the probability density function of the binary hypothesis test is calculated based on the signal-to-noise ratio, Rayleigh fading, and channel deterministic variables.

[0071] Furthermore, the channel deterministic variables are determined based on the reference channel power gain, the distance between the UAV and the edge device, the transmit power of the edge device, and the noise power at the UAV.

[0072] The probability density function is derived through the communication relationship between the signal-to-noise ratio, the Rayleigh fading, and the channel deterministic variables. Specifically, the communication relationship is as follows:

[0073] The latency of federated learning, a key performance characteristic, is affected by the communication model. In the drone scenario, due to the continuous change in motion trajectory, the communication model directly determines the latency of federated learning. In the (t+1)th time slot / round of federated learning, the channel gain between the i-th qualified device and the drone is... Channel gain can be expressed as

[0074]

[0075] In the formula, α represents the reference channel power gain, with a preferred distance of 1 meter. This represents the distance between the drone and the i-th qualified edge device. The denoting factor represents the Rayleigh fading of the drone and the i-th qualified edge device, and β is the path loss exponent from air to ground, typically between 2.5 and 4, depending on the environment.

[0076] in, This represents the calculation process of the distance, where, This represents the distance between the drone and the i-th qualified edge device. Indicates the location of the drone. This represents the position of the i-th qualified edge device. The preferred definition of edge device position is as follows. The preferred positions of the edge devices include the x-axis position, y-axis position, and z-axis position.

[0077] The Rayleigh fading follows an exponential distribution, and the rate of Rayleigh fading is determined by λ. i Characterization, i.e., the function value of Rayleigh fading. Follow the formula below: Therefore, the signal-to-noise ratio (SNR) received by the i-th qualified edge device at the drone can be expressed by the following formula:

[0078]

[0079] in, For signal-to-noise ratio, These are channel deterministic variables, where α represents the reference channel power gain and the value when the preferred distance is 1 meter. p represents the distance between the drone and the i-th qualified edge device. i,t This represents the transmission power of device i in round t. This represents the noise power at the drone's location.

[0080] Similarly, the signal-to-noise ratio (SNR) received by malicious device j at the drone can also be characterized by the same SNR formula, denoted as .

[0081] Qualified edge device i to drone at bandwidth b iThe communication data rate at that time can be expressed as:

[0082]

[0083] In the formula, b represents the communication data rate. i Indicates bandwidth. It is a channel deterministic variable. This represents the Rayleigh fading of the drone and the i-th qualified edge device.

[0084] Specifically, the physical layer authentication model based on signal-to-noise ratio (SNR) difference: To determine the signal source, the UAV deploys a physical layer authentication module. The UAV can utilize the SNR difference between time slot t and time slot (t+1, which can be viewed as global round t and global round t+1 in federated learning), denoted as Γ as described above. t and Γ t+1 The difference. Assume the edge device in the t-th transmission has been certified as a qualified device, i.e. In the formula Represents all qualified edge devices, based on observed Γ t+1 A binary hypothesis test is used to determine whether the source of the (t+1)th transmission is a qualified device. The specific binary hypothesis test is as follows:

[0085]

[0086] In this context, the null hypothesis H0 indicates that the parameters transmitted in the (t+1)th time slot originate from the i-th qualified edge device, while the alternative hypothesis H1 indicates that the parameters transmitted in the (t+1)th time slot originate from the Byzantine device (malicious device) j, and Γ represents the signal-to-noise ratio.

[0087] Therefore, the binary hypothesis testing of drones for edge devices is divided into two cases. The probability density function of the binary false test includes the probability density function of qualified edge devices and the probability density function of unqualified edge devices. The derivation process of the probability density function of qualified edge devices is as follows:

[0088] In round t+1 of global training, if the parameters are uploaded by qualified edge devices, the signal-to-noise ratio difference between adjacent transmissions can be re-derived as follows:

[0089] In the formula, v represents the signal-to-noise ratio difference of qualified edge devices. These are the channel deterministic variables for round t+1 and round t, respectively. Let represent the Rayleigh fading of the drone and the i-th legitimate edge device in round t+1 and round t, respectively.

[0090] The signal-to-noise ratio difference *v* between legitimate edge devices is used as a random variable in the binary hypothesis test. The probability density function (PDF) for the binary hypothesis test of qualified edge devices can be derived as follows:

[0091]

[0092] The proof process is as follows: based on Rayleigh fading and Given the distribution, we can re-represent v = g1 - g0, where and Substituting this into the probability density function (PDF) of the binary hypothesis test, we can obtain the two probability density functions as follows:

[0093]

[0094] Therefore, we can derive the following proof formula for the probability density function of qualified edge devices:

[0095]

[0096] In the above proof formulas, f represents the Rayleigh fading of the drone and the i-th legitimate edge device. G Let f represent the probability density function of the two parts. V (v) represents the overall pass / fail probability density function, λ i This represents the rate of Rayleigh fading, where v is the difference in signal-to-noise ratio between qualified edge devices. are the channel deterministic variables for round t+1 and round t, respectively, and g1 and g0 are the restated representations of the signal-to-noise ratio difference of qualified edge devices.

[0097] By proving the formula for the probability density function of qualified edge devices, the probability density function of the qualified edge devices can be obtained, thus completing the proof.

[0098] The derivation process of the probability density function of non-conforming edge equipment is as follows:

[0099] Considering that the model parameters are uploaded by malicious nodes, the signal-to-noise ratio difference between malicious edge devices in adjacent transmissions can be expressed as:

[0100]

[0101] In the formula, w represents the signal-to-noise ratio difference between malicious edge devices. These are the channel deterministic variables for round t+1 and round t, respectively. Let represent the Rayleigh fading of the drone and the i-th legitimate edge device in round t+1 and round t, respectively.

[0102] The signal-to-noise ratio difference w of malicious edge devices is used as a random variable in the binary hypothesis test. The probability density function (PDF) of the binary hypothesis test for malicious edge devices can be derived as follows:

[0103]

[0104] The proof process is as follows: According to The distribution, The distribution can be expressed as because and Proof formula for the probability density function of malicious edge devices:

[0105]

[0106] In the above proof formulas, Let represent the Rayleigh fading of the drone and the i-th legitimate edge device in round t+1 and round t, respectively.

[0107] The signal-to-noise ratio difference w, λ between malicious edge devices i λ j These represent the Rayleigh fading rates of qualified and malicious edge devices, respectively. are the channel deterministic variables for round t+1 and round t, respectively, and g1 and g0 are the re-representations of the signal-to-noise ratio difference of malicious edge devices.

[0108] By proving the formula using the probability density function of malicious edge devices, we can obtain the probability density function of the binary hypothesis test for malicious edge devices, thus completing the proof.

[0109] Based on the signal-to-noise ratio received from two consecutive transmissions, the statistic for the binary hypothesis test is defined as follows:

[0110]

[0111] Among them, Π t is the optimal detection threshold, used to determine whether a malicious device participates in transmitting parameters in the (t+1)th time slot. D1 and D2 represent the decision thresholds supporting H0 and H1, respectively. ΔΓ represents the statistic, and Γ is the signal-to-noise ratio.

[0112] By using the probability density functions of qualified edge devices and malicious edge devices, we can further derive the false alarm rate and false negative rate of drones performing physical layer authentication, as well as derive the optimal detection threshold.

[0113] The false alarm rate (FAR) refers to the probability that a local model uploaded by a qualified edge device fails to pass the drone aggregator certification, and can be derived from the following formula:

[0114]

[0115] In the formula, For false alarm rate, This indicates that the drone was identified as a malicious device. This represents the null hypothesis, that the drone is determined to be a qualified edge device, λ. i This represents the rate of Rayleigh fading, where v is the difference in signal-to-noise ratio between qualified edge devices. and These are the channel deterministic variables for the t-th global round and the (t+1)-th global round, respectively. t It is the detection threshold.

[0116] Furthermore, the number of qualified edge devices affects the convergence and performance of federated learning. Due to the high false positive rate, some unselected qualified edge devices may be falsely identified as malicious edge devices. The number of these false identifications can be calculated using the following formula:

[0117]

[0118] In the formula, This indicates the number of qualified edge devices that were mistakenly identified as malicious edge devices. Let J represent the false positive rate, J represent the total number of malicious edge devices, and j represent the number of malicious edge devices, where s t,i Edge device i is identified as a qualified device.

[0119] The false negative rate is determined by a probability function based on Rayleigh fading rate, the signal-to-noise ratio difference of malicious edge devices, and channel deterministic variables, specifically:

[0120] The false negative rate (MDR) refers to the probability that an illegal or malicious device can impersonate a legitimate device, pass authentication, and participate in federated learning. This is a relatively serious situation, and the probability of its occurrence can be derived as follows:

[0121]

[0122] In the formula, λ represents the false negative rate. i λ j Represents the Rayleigh fading rate of a qualified edge device and a malicious edge device, respectively, where w is the signal-to-noise ratio difference of the malicious edge device. and These are the channel deterministic variables for the t-th global round and the (t+1)-th global round, respectively. t It is the detection threshold.

[0123] In reality, malicious edge devices will try their best to maximize the false negative rate. To increase the probability of intrusion, qualified edge devices expect the maximum potential false negative rate to meet the following constraints:

[0124]

[0125] In the formula, ∈ represents the level of inherent security threat with the highest probability of system intrusion. This represents the maximum potential false negative rate for qualified edge devices.

[0126] The process of determining the optimal detection threshold is as follows: In each global round t, the UAV needs to determine the optimal detection threshold to minimize the false alarm rate and the false alarm rate. The specific formula for minimizing the false alarm rate and the false alarm rate is as follows:

[0127]

[0128] In the formula, Indicates the optimal detection threshold, Π t It is the detection threshold, σ t This represents the maximum potential false negative rate for qualified edge devices, and ∈ represents the intrinsic security threat level with the highest probability of system intrusion.

[0129] The proof is as follows: It can be deduced that the false negative rate is related to the detection threshold Π. t The function is monotonically increasing, while the false alarm rate is related to the detection threshold Π. t The monotonically decreasing function is derived as follows:

[0130]

[0131] and,

[0132]

[0133] In the formula, λ i λ j These represent the Rayleigh fading rates of qualified and malicious edge devices, respectively. and These are the channel deterministic variables for the t-th global round and the (t+1)-th global round, respectively. It is the detection threshold.

[0134] Therefore, reducing the false negative rate may increase the false positive rate. The optimal detection threshold can be obtained by minimizing the false positive and false negative rates using the aforementioned formula, and the optimal solution is obtained when the equilibrium point is reached.

[0135] Furthermore, the UAV orchestration resource model includes an objective function and constraints; the objective function is to minimize the security learning cost function based on UAV trajectory, federated learning participation variables, and latency / energy consumption variables; the constraints include a comparison between the false negative rate and the security threat threshold, a minimum participating device constraint for federated learning based on the federated learning participation variables, and UAV motion constraints based on the UAV trajectory. The latency / energy consumption variables include bandwidth allocation, computation frequency, and transmission power.

[0136] Furthermore, the LSTM-SDRL algorithm includes an LSTM sub-algorithm and an SDRL sub-algorithm;

[0137] The LSTM sub-algorithm is used to strengthen the expected cumulative reward network and the network for increasing the degree of violation of security constraints, thereby generating a strengthened expected cumulative reward network and a strengthened network for increasing the degree of violation of security constraints.

[0138] The SDRL sub-algorithm constructs a policy function based on the enhanced expected cumulative reward network and the enhanced security constraint violation degree network to solve for and generate the UAV orchestration policy.

[0139] Furthermore, the hidden state of the output layer of the LSTM sub-algorithm is connected to the SDRL sub-algorithm through a fully connected neural network.

[0140] Furthermore, the policy distribution function of the SDRL sub-algorithm is constructed based on the expected cumulative reward network, the expected degree of security constraint violation network, and the reward function.

[0141] Furthermore, the SDRL sub-algorithm constructs a reward function based on the negative penalty constraint of safe learning cost and false negative rate.

[0142] The above solution enables UAVs to perform efficient and reliable authentication of edge devices, achieve Pareto optimality in training accuracy, latency, and energy consumption, and provide four guarantees for UAVs to carry out edge device data mining tasks: security performance, communication performance, energy-saving performance, and model accuracy.

[0143] Latency and energy consumption analysis: The latency and energy consumption of each edge device mainly involve the local computing phase and transmission phase in the federated learning training process.

[0144] Local computation phase: In time slot t, the local training latency of the i-th edge device can be expressed as:

[0145]

[0146] Among them, C i,t f represents the computational complexity of training a sample dataset using the backpropagation algorithm on the i-th edge device. i,tThis represents the CPU frequency of the i-th device. D represents the local training latency. i,t This represents the size of the dataset for edge device i.

[0147] During the local computation phase, energy consumption can be expressed as:

[0148]

[0149] in, f represents energy consumption. i,t ζ represents the CPU frequency of the i-th device. i C is a calculation coefficient related to the CPU of the i-th device. i,t This represents the computational complexity of training the i-th edge device using the backpropagation algorithm on a single sample dataset. Indicates the number of local training rounds.

[0150] Transmission Phase: Given fixed dimensions for both the global and local models, the amount of data each edge device must send to the drone's local model parameters in each iteration is denoted as a constant. Therefore, the latency of the transmission phase can be expressed as:

[0151]

[0152] In the formula, This indicates the latency during the local transmission phase. Ω represents the communication data rate, and Ω represents the size of the model.

[0153] Accordingly, the energy consumption of the i-th device during the transmission phase can be expressed as:

[0154]

[0155] In the formula, Indicates energy consumption. p represents the latency during the local transmission phase. i,t This represents the transmit power of edge device i.

[0156] The energy consumption of a drone includes motion and parameter aggregation. Since the latency is relatively low compared to transmission and local computation, the energy consumption during aggregation can be ignored. Motion power consumption can be calculated as follows:

[0157]

[0158] This formula, by incorporating key physical quantities during flight, such as flight distance, time, rotor speed, air resistance, and the drone's static power consumption, accurately reflects the complex changing trends of the drone's flight power. Indicates the power consumption during motion, P0 and P1 represent the propeller profile power and thrust power of the UAV when hovering, respectively, r t r t-1 Let τ represent the positions of the UAV in round t and round t-1, respectively. t U represents the total delay. tip ρ is the tip velocity of the blade, v0 represents the average rotor induced velocity during hovering, and d0, ρ0, s0, and A correspond to the fuselage drag ratio, air density, rotor solidity, and rotor disk area, respectively.

[0159] In summary, the total latency of a global round can be expressed as follows:

[0160]

[0161] In the formula, τ t Indicates the total delay. These represent the latency of the local computation phase and the transmission phase, respectively.

[0162] Furthermore, the total energy consumption of all devices in the t-th global round can be expressed as:

[0163]

[0164] In the formula, E t Indicates total energy consumption. These represent the total energy consumption of the local computing phase and the transmission phase, respectively.

[0165] To meet the diverse needs of different federated learning methods, such as latency-sensitive or energy-sensitive methods, we utilize training cost TC to achieve a balance between latency and energy, namely:

[0166]

[0167] In the formula, TC t For training costs, τ t For the total delay, E represents the weighting coefficient. t This indicates the total energy consumption.

[0168] Meanwhile, the system proposed in this embodiment needs to maximize the number of security devices while reducing the false alarm rate and limiting the false alarm rate. To this end, a new metric is redefined: Security Learning Cost (SLC).

[0169]

[0170] Among them, Ψ t τ represents the cost of safe learning. t For the total delay, E represents the weighting coefficient.t Represents total energy consumption, denominator The participation level is represented by i, where i represents the qualified edge device, I represents the total number of qualified edge devices, and k represents the number of qualified edge devices. t,i Let j represent the participants in federated learning, and J represent all malicious edge devices. This indicates the false negative rate. This metric takes into account both the actual number of participants and the level of false positives.

[0171] The optimization problem for drones: After establishing the above models, from the drone's perspective, its mission objective is to minimize latency and energy consumption while ensuring the participation of qualified equipment, all while keeping the system intrusion level within a certain range. Therefore, our optimization problem and constraints are as follows:

[0172]

[0173] In the formula, P1 is the objective function for minimizing UAV orchestration resources (computing resources, communication resources), and C1 to C7 are seven constraints. The objective function includes minimizing latency and energy consumption, and maximizing device participation. k t ,b t ,f t ,p t These include the drone trajectory, federated learning participation variables, bandwidth allocation, computation frequency, and transmission power. The drone trajectory... Represents the trajectory design of the drone, and the variables involved in federated learning. It is a binary variable representing whether the i-th qualified edge device is selected in the t-th round, and the bandwidth allocation. Calculate frequency Transmission power

[0174] Constraint (C1) guarantees the minimum detection rate (MDR), i.e., the intrusion threat to the system. Constraint (C2) is a minimum participating device constraint based on the federated learning participation variables, ensuring the minimum participating device. Constraint (C3) restricts the physical motion of the UAV, and constraints (C4) to (C6) are UAV motion constraints based on the UAV trajectory, limiting wireless resources. Finally, constraint (C7) ensures that the UAV's energy consumption does not exceed the energy budget.

[0175] The LSTM-SDRL algorithm addresses a non-convex, multi-step decision problem where the objective function P1 requires online state information from the environment. Furthermore, due to potential intrusion from malicious edge devices, the drone agent needs to explore the action space while avoiding dangerous policies that could disrupt the federated learning task. Therefore, the LSTM-SDRL approach is employed to solve this problem. The objective function P1 is transformed into a constrained Markov decision process (CMDP), and a secure deep reinforcement learning (SDRL) environment is explicitly defined. The state space, action space, reward, and cost in the model are described below.

[0176] State space: In the t-th round of global federated learning, state s t ∈S contains the following information: the drone's position in round t-1, and the positions r of qualified and malicious edge devices. I,t-1 ,r J,t-1 The power p transmitted by the malicious edge device in round t J,t Dataset size D t The computational complexity is C. t In addition, it also includes the remaining energy E of the drone in round t. u,t The state space can be summarized as follows:

[0177] s t ={r u,t-1 ,r I,t-1 ,r J,t-1 ,p J,t C t D t E u,t}

[0178] In the formula, state s t Representing the state space, r u,t-1 ,r I,t-1 ,r J,t-1 ,p J,t C t D t E u,t Represent the location, transmission power, constraints, dataset size, and remaining energy E of the drone in round t for qualified and malicious edge devices, respectively. u,t .

[0179] Action space: After observing its state, the drone will react, and its action space is defined as follows:

[0180] a t ={r u,t ,c t ,p t ,f t ,b t}

[0181] In the formula, a t Represents the action space, r u,t ,c t ,p t ,f t ,b t These represent the drone trajectory, federated learning participants, transmission power, computation frequency, and bandwidth allocation, respectively.

[0182] Reward Function: Based on the objective function and constraints of the optimization problem, the reward designed in this embodiment aims to minimize the safe learning cost of the entire Federated Learning (FL) process. The reward can be designed as follows:

[0183]

[0184] Where -Ψ is the negative value of the cost of secure learning, and This represents a negative penalty constraint based on the constraints when the missed detection rate (MDR) exceeds a threshold, the drone violates motion patterns, and the energy budget. All three penalties are scaled by multiplying by their respective weights.

[0185] Cost: According to the definition of secure reinforcement learning, the cost designed in this embodiment is based on reducing the false detection rate to evaluate the risk value of the policy, thereby avoiding dangerous exploration. Specifically, the selected security constraints... It is quantified into M risk levels.

[0186]

[0187] Algorithm Framework: The proposed LSTM-SDRL framework comprises an expected cumulative reward network Q and a safety constraint violation expectation network E, hereinafter referred to as the Q network and E network. Both the Q network and E network consist of an online network and a target network. The Q network estimates the state-action value function Q(s,a), which represents the expected cumulative reward given state s and action a. The E network estimates the risk or penalty associated with potential safety constraint violations; therefore, the function E(s,a) represents the expected degree of safety constraint violation. The target network is a copy of the online network, used to improve training efficiency and stability. Furthermore, historical data is stored in a replay buffer. Subsequently, a subset of these data is randomly selected from the buffer to train the neural network, thereby significantly reducing the correlation between data points.

[0188] LSTM Module: To capture temporal patterns of environmental dynamics (e.g., time-varying federated learning tasks and observed malicious power transmitted by Byzantine nodes), this invention introduces an LSTM network, a recurrent neural network specifically designed for processing and predicting time series data. The input to the LSTM is time slot t and a sequence of historical states observed in the previous seq-1 time slots. The output of the LSTM module is the hidden state h. t This state will be passed to the secure deep reinforcement learning part through a fully connected neural network (dense layers). Specifically, the LSTM module consists of a forget gate f t Input gate i t Output gate o t Unit door g t and unit state C t Composition can be expressed as:

[0189] i t =σ(W i [s t ,h t-1 ]+b i ),

[0190] f t =σ(W f ·[s t ,h t-1 ]+b f ),

[0191] g t =tanh(W c ·[s t ,h t-1 ]+b c )

[0192] C t =f t *C t-1 +i t *g t ,

[0193] o t =σ(W o ·[s t ,h t-1 ]+b o )

[0194] h t =o t *tanh(C t ).

[0195] SDRL Algorithm Framework: In this embodiment, the neural network structure of SDRL mainly includes a Q-network, a target Q-network, an E-network, and a target E-network. The output value of the Q-network evaluates the long-term expected reward for each state-action pair. The target Q-value is calculated as follows:

[0196]

[0197] Among them, the learnable weight ω measures the importance of future rewards, which are evaluated by the target Q network.

[0198] The E-value describes the risk of each state-action pair, and it is updated according to the reward function formula based on L future risk values, as shown below:

[0199]

[0200] Here, the discount rate χ represents the expected value of future risk.

[0201] Unlike the traditional Boltzmann distribution, the policy distribution of state-action pairs in secure deep reinforcement learning is based not only on the Q-value but also on the E-value, which can be obtained by the following formula:

[0202]

[0203] In the formula, Υ weighs the importance of long-term expected rewards, π(s) t ,a t ) is the policy distribution function.

[0204] See the training process. Figure 3 First, initialize the LSTM-enhanced Q-network and E-network, parameterized by parameters θ and φ, respectively (line 1). Then, initialize the target Q-network and E-network by replicating the above online network (line 2). Additionally, initialize the replay buffer D to store historical samples (line 3).

[0205] During the exploration phase, the agent observes state s. t And construct a state sequence (Line 6). Then, the Q-network and E-network output the Q-value and E-value evaluations for each state-action pair (Lines 7-8). Next, the agent selects an action according to formula (40) (Line 9). After executing the action, the agent enters the next state s. i+1 and obtain rewards from the environment. i (Line 10). Meanwhile, in SDRL, the agent also evaluates the risk c. t (Line 11). Following this, the sample data quintuples... It will be saved to the playback buffer D for later retrieval.

[0206] During the training phase (lines 14-21), mini-batch transformations of size H are randomly sampled from D to train the Q and E networks. Furthermore, the target Q and E networks are updated once every set number of steps using a soft update method.

[0207] In summary, a drone-assisted physical layer authentication-enhanced federated learning method was designed. This method achieves Pareto optimality in training accuracy, latency, and energy consumption while ensuring efficient and reliable authentication of the client (edge ​​device) by the server (drone). This provides a quadruple guarantee for drones conducting edge device data mining tasks, improving security, communication performance, energy efficiency, and model accuracy.

[0208] In this embodiment, malicious devices with ineligible authentication are identified at the physical layer, ensuring the security, communication performance, energy efficiency, model accuracy, high privacy, and low communication overhead of federated learning. This satisfies the requirement for UAVs to perform efficient and reliable authentication of edge devices, achieving Pareto optimality in training accuracy, latency, and energy consumption. This provides a quadruple guarantee for UAVs conducting edge device data mining tasks, enhancing security, communication performance, energy efficiency, and model accuracy.

[0209] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0210] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A federated learning method for unmanned aerial vehicles (UAVs) with enhanced physical layer authentication, characterized in that, Drones, acting as mobile servers, participate in federated learning with ground-based edge devices, including: The authentication of the edge device is determined by a communication model based on physical layer authentication enhancement. If the authentication is successful, the drone receives the local model updated by the federated learning of the edge device. If the authentication is unsuccessful, the drone refuses to receive the local model. The communication model is a binary hypothesis testing model using signal-to-noise ratio and false negative rate. Based on the false negative rate, a UAV orchestration resource model is constructed. The UAV orchestration resource model is solved using the LSTM-SDRL algorithm to generate a UAV orchestration strategy to determine the security performance, communication performance, energy-saving performance and model accuracy performance. The drone orchestration resource model mentioned above includes an objective function and constraints. The objective function is to minimize the safety learning cost function based on UAV trajectory, federated learning participation variables, and latency and energy consumption variables. The constraints include a comparison between the false negative rate and the security threat threshold, a minimum participating device constraint for federated learning based on the federated learning participation variables, and a drone motion constraint based on the drone trajectory. The latency and energy consumption variables include bandwidth allocation, computing frequency, and transmission power. The federated learning participation variables, bandwidth allocation, computing frequency, transmission power, and false negative rate are determined based on whether the edge device's authentication is successful. The LSTM-SDRL algorithm includes an LSTM sub-algorithm and an SDRL sub-algorithm. The state space of the SDRL sub-algorithm includes the location of the edge device with valid authentication, the location of the edge device with invalid authentication, the transmission power of the edge device, constraints, dataset size, and the remaining energy of the UAV. The action space includes the UAV trajectory, federated learning participation variables, edge device transmission power, computation frequency, and bandwidth allocation. The reward function is determined based on the security learning cost function and the constraints. The LSTM sub-algorithm is used to initialize the network parameters of the Q-network and E-network, which are used to calculate the state space and action space.

2. The physical layer authentication-enhanced UAV federated learning method according to claim 1, characterized in that, The local optimization problem of the communication model based on physical layer authentication enhancement aims to minimize the local loss function of the model variation between the global model and the local model of the edge device in the time slot. The local optimization problem includes a local variation loss term, a local loss function gradient term, and a global loss function gradient term. The local variation loss term is a local loss function with the sum of the global model and model variations as variables; The local loss function gradient term includes the gradient of the local loss function in the global model; The gradient term of the global loss function includes the gradient of the global loss function in the global model.

3. The physical layer authentication-enhanced UAV federated learning method according to claim 1, characterized in that, The communication model performs a binary hypothesis test based on the difference in signal-to-noise ratio between adjacent global rounds of federated learning to determine whether the authentication of the edge device is qualified. The probability density function of the binary hypothesis test is calculated based on Rayleigh fading and channel deterministic variables.

4. The physical layer authentication-enhanced UAV federated learning method according to claim 3, characterized in that, The channel deterministic variables are determined based on the reference channel power gain, the distance between the UAV and the edge device, the transmit power of the edge device, and the noise power at the UAV. The probability density function is derived from the communication relationship between the signal-to-noise ratio, the Rayleigh fading, and the channel deterministic variables.

5. The physical layer authentication-enhanced UAV federated learning method according to claim 1, characterized in that, The false negative rate is determined by a probability function based on Rayleigh fading rate, the difference in signal-to-noise ratio between malicious edge devices, and channel deterministic variables.

6. The physical layer authentication-enhanced UAV federated learning method according to any one of claims 1 to 5, characterized in that, The hidden state of the output layer of the LSTM sub-algorithm is connected to the SDRL sub-algorithm through a fully connected neural network.

7. The physical layer authentication-enhanced UAV federated learning method according to any one of claims 1 to 5, characterized in that, The policy distribution function of the SDRL sub-algorithm is constructed based on the expected cumulative reward network, the network of the degree of violation of security constraints, and the reward function.

8. The physical layer authentication-enhanced UAV federated learning method according to any one of claims 1 to 5, characterized in that, The SDRL sub-algorithm constructs a reward function based on the negative penalty constraint of safe learning cost and false negative rate.

Images