Edge Computing and Information Transmission Method and System for Power Grid Equipment Based on EKF-PPO Algorithm

Abstract

A method and system for edge computing and information transmission of power grid equipment based on the EKF-PPO algorithm. The method includes: constructing a state evolution model of a mobile eavesdropping device; constructing a radar measurement model; predicting the state of the mobile eavesdropping device and the MSE matrix based on the state evolution model; calculating the channel between the UAV and the mobile eavesdropping device, and using the PPO algorithm to design the trajectory and speed of the UAV, the number of bits for local computing of the edge device and the number of bits uploaded to the UAV, the transmit beamforming vector of the edge device, and the frequency of local computing of the edge device; calculating the Kalman gain using the radar measurement noise, and correcting the state of the mobile eavesdropping device and the MSE matrix based on the Kalman gain and real-time radar measurement data. The present invention uses the EKF algorithm to estimate the position of the mobile eavesdropping device in real time, ensuring that the offloaded data can avoid being exposed to the eavesdropping device. At the same time, the PPO algorithm is used to optimize the flight trajectory, communication beam, and data offloading decision of the UAV, ensuring the security and efficiency of signal transmission.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technologies, and in particular, to a method and system for power grid device edge computing and information transmission based on the EKF-PPO algorithm. Background Art

[0002] Power equipment in remote mountainous areas usually includes transformers, circuit breakers, distribution boxes, solar power generation equipment, wind power generation equipment, and power load regulators, etc. Due to the remote geographical location, the maintenance and monitoring of these devices are difficult. Especially in the case of incomplete network signal coverage or poor communication conditions, traditional data transmission methods may not be able to meet the requirements of real-time monitoring. Each power device collects data through sensors installed near the device, and these data involve multiple aspects such as voltage, current, temperature, and device operating status. With the support of edge computing devices, real-time data can be processed and analyzed near the device to facilitate the upload of important information.

[0003] Drones are deployed in mountainous areas as temporary or regular mobile base stations, responsible for collecting real-time data from different power devices. Drones have high mobility and can fly to the vicinity of different power devices as needed, receive the data uploaded from edge computing devices, and perform relay forwarding. The edge computing unit near the power device is responsible for real-time data collection and preprocessing. For example, the edge device can analyze data such as the temperature, voltage, and current of the device to determine whether there is a risk of failure. If the device is in a normal state, the edge device will regularly upload the data to the drone; if an abnormality of the device is detected (such as too high temperature, unstable current, etc.), the edge device will immediately upload an alarm message to ensure a quick response. Edge computing enables the local processing of device data, reducing data transmission latency and bandwidth pressure. At the same time, in an environment with unstable wireless signals or limited bandwidth, edge computing helps to reduce the amount of data transmitted and ensure the efficient use of the communication link. As a base station, the drone has strong communication capabilities. It can receive the data transmitted by the edge computing modules of multiple power devices and upload the data to the remote monitoring center through a stronger network (such as satellite communication, LTE, 5G, etc.). The flight altitude and flexibility of the drone enable it to cross the obstacles in the mountainous area to ensure stable communication with remote devices.

[0004] Due to the possible existence of various electromagnetic interferences or threats of external attacks (such as eavesdropping, signal jamming, etc.) in the mountainous environment, the security of wireless communication is of crucial importance. Therefore, strong encryption technologies (such as AES, VPN tunnels, etc.) are adopted for the communication between the UAV and the edge device to ensure that the data is not stolen or tampered with during the transmission process. The UAV is also equipped with anti-jamming functions, which can automatically detect and respond to signal jamming to ensure the stability of the communication link. If a jamming source is detected, the UAV can automatically adjust its flight path or switch to an alternative frequency band to avoid the interference. However, the uncertainty of the position of the mobile eavesdropping device poses a great challenge to the secure transmission of information of the power grid equipment. Therefore, in order to ensure the secure transmission of information of the power grid equipment, it is necessary to obtain the accurate and real-time position of the eavesdropping device before the power grid equipment uploads information data. Summary of the Invention

[0005] The objective of the present invention is to overcome the above-mentioned defects and problems existing in the prior art, and provide a power grid equipment edge computing and information transmission method and system based on the EKF-PPO algorithm. By using the EKF algorithm, the position of the mobile eavesdropping device is estimated in real time to ensure that the offloaded data can be avoided from being exposed to the eavesdropping device. At the same time, by using the PPO algorithm, the flight trajectory, communication beam, and data offloading decision of the UAV are optimized to ensure the security and efficiency of signal transmission.

[0006] To achieve the above objective, the technical solution of the present invention is: A power grid equipment edge computing and information transmission method based on the EKF-PPO algorithm, including:

[0007] Construct a state evolution model of the position and speed of the mobile eavesdropping device;

[0008] Based on the echo signal received by the UAV, construct a radar measurement model;

[0009] Based on the state evolution model, predict the state of the mobile eavesdropping device and the MSE matrix;

[0010] Calculate the channel between the UAV and the mobile eavesdropping device, and use the PPO algorithm to design the trajectory and speed of the UAV, the number of bits for local computing of the edge device and the number of bits uploaded to the UAV, the transmit beamforming vector of the edge device, and the frequency of local computing of the edge device;

[0011] Calculate the Kalman gain using the radar measurement noise, and correct the state of the mobile eavesdropping device and the MSE matrix based on the Kalman gain and real-time radar measurement data.

[0012] The state evolution model of the mobile eavesdropping device is:

[0013] ; ;

[0014] ; ; ;

[0015] wherein, is a time slot; is a state vector; is a linear state evolution matrix; is state evolution noise; is the abscissa of the mobile eavesdropping device; is the ordinate of the mobile eavesdropping device; is the component of the velocity of the mobile eavesdropping device on the axis; is the component of the velocity of the mobile eavesdropping device on the axis; is the time slot length; is state evolution error of; is state evolution error of; is state evolution error of; is state evolution error of; is covariance matrix of; is variance of; is variance of; is variance of; is variance of.

[0016] The radar measurement model is:

[0017] ; ;

[0018] ; ;

[0019] ; ;

[0020] wherein, is a measurement vector; is the measured value of the elevation angle of the UAV; is the measured value of the azimuth angle of the UAV; is the measured value of the Doppler frequency shift; is the measured value of the time delay; is the true value of the time delay; is the coordinate of the UAV; Are the coordinates of the mobile eavesdropping device; Is the speed of light; Is the true value of the Doppler frequency shift; Is the speed of the mobile eavesdropping device; Is the wavelength of the signal; Is the measurement noise; Is the measurement error of the elevation angle; Is the measurement error of the azimuth angle; Is the measurement error of the time delay; Is the measurement error of the Doppler frequency shift; Is The covariance matrix of; Is the covariance matrix of the elevation angle and the azimuth angle; Is the variance of the time delay; Is the variance of the Doppler frequency shift;

[0021] The measurement vector And the state vector The relationship of Is expressed as:

[0022] ; ;

[0023] ;

[0024] In the formula, Is the true value of the elevation angle; Is the ordinate of the mobile eavesdropping device; Is the ordinate of the UAV; Is the abscissa of the mobile eavesdropping device; Is the abscissa of the UAV; Is the flight altitude of the UAV;

[0025] By introducing variables And , Using the Cramer-Rao lower bounds of the elevation angle and the azimuth angle to calculate the covariance matrix of the elevation angle and the azimuth angle, which is expressed as:

[0026] ; ;

[0027] ;

[0028] In the formula, And Are respectively And The Cramer-Rao lower bounds of; Is the number of received signal samples; Is the number of receiving antennas; is the transmit signal-to-noise ratio; is the edge device - large-scale fading of the sensing channel of the mobile eavesdropping device - UAV; is from the UAV to the edge device transmit antenna steering vector; is the edge device transmit beamforming vector; is the signal propagation noise;

[0029] For the time delay and Doppler frequency shift the Cramer-Rao lower bound is calculated as:

[0030] ; ;

[0031] ; ;

[0032] ;

[0033] In the formula, is the bandwidth of the signal; is the receive signal-to-noise ratio; is the time slot length; is the sampling rate of the analog-to-digital converter; is the edge device - mobile eavesdropping device - sensing channel of the UAV; is the receive matching filter; is from the UAV to the edge device receive antenna steering vector.

[0034] Based on the state evolution model, predict the state of the mobile eavesdropping device at the th time slot:

[0035] ;

[0036] By calculating the Jacobian matrix, approximate the non-linear radar measurement model as a linear measurement model:

[0037] ;

[0038] In the formula, is the Jacobian matrix of; is the prior estimate of the state of the mobile eavesdropping device at the th time slot;

[0039] Estimate the The prior estimate of the MSE matrix for each time slot is as follows:

[0040] ;

[0041] Wherein, is the MSE matrix for the th time slot; is the covariance matrix of the state evolution noise .

[0042] The channel between the UAV and the mobile eavesdropping device is:

[0043] ;

[0044] Wherein, is the channel between the UAV and the mobile eavesdropping device; is the channel gain at a relative distance of 1 m; is the distance between the UAV and the mobile eavesdropping device; is the receiving antenna steering vector pointing from the UAV to the mobile eavesdropping device; is the transmitting antenna steering vector pointing from the UAV to the mobile eavesdropping device; is the Hermitian conjugate of the vector.

[0045] Calculate the Kalman gain using the radar measurement noise:

[0046] ;

[0047] Wherein, is the Kalman gain; is the Jacobian matrix of; is the covariance matrix of;

[0048] Using the Kalman gain and real-time radar measurement data, correct the state prediction to:

[0049] ;

[0050] Wherein, is the posterior estimate of the state of the mobile eavesdropping device; is the prior estimate of the state of the mobile eavesdropping device; is the measurement vector; represents the relationship between and

[0051] The MSE matrix is updated to: ;

[0052] Wherein, is the identity matrix; is the prior estimate of the MSE matrix.

[0053] The data interaction process between grid edge devices and drones is represented as the following Markov decision process model:

[0054] The -step state is: ;

[0055] In the formula, is the state space of the Markov decision process model; is the channel between the edge device and the drone; is the channel between the edge device and the mobile eavesdropping device; is the sensing channel of the edge device -mobile eavesdropping device-drone; is the data volume of the edge device ; is the position of the drone; is the position of the mobile eavesdropping device;

[0056] The -step action is:

[0057] ;

[0058] In the formula, is the action space of the Markov decision process model; is the normalized value of the drone transmit beamforming vector; is the normalized value of the CPU clock frequency of the edge device; is the ratio of the offloaded data volume to the total data volume; is the normalized value of the magnitude of the drone's acceleration; is the normalized value of the direction of the drone's acceleration;

[0059] The speed of the drone, the transmit beamforming vector of the edge device, and the frequency of local computing of the edge device are recovered through the following formulas:

[0060] ;

[0061] ;

[0062] ; ;

[0063] In the formula, is the component of the drone speed on the axis; is the drone speed on the Component of the axis is the maximum speed of the UAV in each time slot; is the normalized value of the magnitude of the UAV acceleration; is the time slot length; is the maximum acceleration of the UAV in each time slot; is the UAV speed in the th time slot; is the normalized value of the direction of the UAV acceleration; is the edge device in the waveform shaping design in the th time slot; is the maximum transmission power of the UAV; is the edge device in the normalized value of the waveform shaping design in the th time slot; is the number of transmitting antennas of the UAV; is the edge device CPU clock frequency; is the edge device normalized value of the CPU clock frequency; is the maximum CPU clock frequency of the edge device; is the minimum CPU clock frequency of the edge device;

[0064] The reward is:

[0065] ;

[0066] ; ;

[0067] ; ;

[0068] ; ;

[0069] In the formula, is the function obtained by performing the action in the th step; is the number of edge devices; is the energy consumed by the edge device while processing data; is the energy consumed by the edge device while uploading data; is the data processing time penalty function; is the data upload time penalty function; is the penalty function weight; , is the penalty function weight; is the edge device 's safety rate; is the data processing time; is the data upload time; is the threshold of data processing time and upload time; is the effective capacitance coefficient; is the edge device 's CPU frequency; is the edge device the ratio of the offloaded data volume to the total data volume; is the number of bits of the total data volume; is the edge device the number of CPU cycles required to process 1 bit of data; is the bandwidth of the signal.

[0070] Input the state into the Actor network and the Critic network, and output the action and the state value :

[0071] The Actor network adopts a 4-layer fully connected network, with the state as the input, and outputs the mean vector of the action and the standard deviation vector , and the action policy is:

[0072] ;

[0073] In the formula, is the new action policy; is the action at the th step; is the state; is the parameter of the Actor network;

[0074] The loss function of the actor is:

[0075] ;

[0076] ; ;

[0077] In the formula, is the probability ratio of the new action policy to the old action policy; is the old action policy; is the advantage function; is the action value function; is the state value;

[0078] Adopt the generalized advantage estimation method to estimate :

[0079] ; ;

[0080] In the formula, is the temporal difference; is the discount factor; is the maximum number of steps;

[0081] The clipping method is adopted to process the loss function as:

[0082] ;

[0083] ;

[0084] In the formula, is the processed loss function; is the clipping factor; is the clipping function;

[0085] The Critic network adopts a 4-layer fully connected network, with the state as the input, and outputs the state value . The objective function for updating the Critic network is:

[0086] ; ;

[0087] In the formula, is the loss function of the Critic network; is the cumulative discounted reward; is the function obtained by performing the action at the th step; is the time period.

[0088] A power grid equipment edge computing and information transmission system based on the EKF-PPO algorithm, which is applied to the method described above. The system includes:

[0089] A state evolution model construction module, which is used to construct a state evolution model of the position and speed of a mobile eavesdropping device;

[0090] A radar measurement model construction module, which is used to construct a radar measurement model based on the echo signal received by the unmanned aerial vehicle;

[0091] A mobile eavesdropping device state prediction module, which is used to predict the state of the mobile eavesdropping device and the MSE matrix based on the state evolution model;

[0092] The PPO algorithm module is used to calculate the channel between the UAV and the mobile eavesdropping device, and design the trajectory and speed of the UAV, the number of bits calculated locally by the edge device and the number of bits uploaded to the UAV, the transmit beamforming vector of the edge device, and the frequency of local calculation of the edge device by using the PPO algorithm;

[0093] The mobile eavesdropping device position correction module is used to calculate the Kalman gain by using the radar measurement noise, and correct the state of the mobile eavesdropping device and the MSE matrix based on the Kalman gain and the real-time radar measurement data.

[0094] A grid device edge computing and information transmission device based on the EKF-PPO algorithm includes a memory and a processor;

[0095] The memory is used to store computer program code and transmit the computer program code to the processor;

[0096] The processor is used to execute the above-mentioned method according to the instructions in the computer program code.

[0097] Compared with the prior art, the beneficial effects of the present invention are:

[0098] In the edge computing and information transmission method and system of grid devices based on the EKF-PPO algorithm of the present invention, the position of the mobile eavesdropping device is estimated in real time based on the extended Kalman filter (EKF) algorithm to ensure that the offloaded data can be avoided from being exposed to the eavesdropping device; at the same time, the trajectory and speed of the UAV, the number of bits calculated locally by the edge device and the number of bits uploaded to the UAV, the transmit beamforming vector of the edge device, and the frequency of local calculation of the edge device are optimized and designed by using the proximal policy optimization-based deep reinforcement learning algorithm (PPO) to ensure the security and efficiency of signal transmission. Description of the Drawings

[0099] Figure 1 is a flowchart of the edge computing and information transmission method of grid devices based on the EKF-PPO algorithm of the present invention.

[0100] Figure 2 is a system block diagram of the UAV and the edge device in the embodiment of the present invention.

[0101] Figure 3 is a flowchart of the EKF algorithm in the embodiment of the present invention.

[0102] Figure 4 is a schematic diagram of the interaction process between the agent and the environment in the PPO algorithm in the embodiment of the present invention.

[0103] Figure 5 is a schematic diagram of the training process of the PPO algorithm in the embodiment of the present invention.

[0104] Figure 6 It is a schematic diagram of the Actor network structure in the embodiment of the present invention.

[0105] Figure 7 It is a schematic diagram of the Critic network structure in the embodiment of the present invention.

[0106] Figure 8 It is a block diagram of the structure of a power grid equipment edge computing and information transmission system based on the EKF-PPO algorithm of the present invention.

[0107] Figure 9 It is a block diagram of the structure of a power grid equipment edge computing and information transmission device based on the EKF-PPO algorithm of the present invention. Detailed implementation manners

[0108] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0109] As Figure 2 shown, consider a drone-assisted edge computing system, which includes drones, M edge devices with N antennas and a potential single-antenna mobile eavesdropping device. Among them, the drone equipped with M antennas also serves as a mobile base station and is equipped with an edge computing server to provide computing services for the K edge devices on the ground. Assume that the drone in the system flies within a limited time period T, and divides the time period into L time slots, that is: ; where is the time span of each time slot.

[0110] In a drone-assisted edge computing system, considering the computing resources and energy limitations of edge devices, these devices usually divide the tasks to be processed into two parts: one part is processed locally, and the other part is offloaded to the edge server on the drone for computing. Among them, during the local computing process, each edge device utilizes local hardware resources and time slot resources to complete the computing tasks. Then, during the offloading process, the edge devices offload the computing tasks assigned to the drone to the drone by designing their respective transmission beams. It should be noted that when designing the transmission beams at the device end, the beam design needs to be carried out according to the position information of the eavesdropping device feedback by the drone to avoid the eavesdropping device illegally obtaining the communication information between the edge device and the drone. In addition to communicating with the edge devices, the drone also needs to locate and track the eavesdropping device in real time, and then send its position information to the edge device through the downlink of the drone and the edge device. To avoid mutual interference between each link, it is considered that the drone adopts the OFDM scheme, and different frequency bands are allocated for the uplink and downlink communication links between the drone and the edge device and the sensing link. In addition, it is assumed that the drone can obtain the channel state information from itself to the edge device to coordinate the sensing beam, local computing, and offloading processes.

[0111] The entire system is placed in a three-dimensional Cartesian coordinate system. Assuming that the position of the edge device does not change, the position of the edge device is represented as ; assuming that the eavesdropping device moves on the ground, and its position at time slot is . Assuming that the flight altitude of the drone is fixed, its coordinates at the th time slot can be represented as , and its movement is subject to the following constraints:

[0112] ;

[0113] wherein, is the initial position of the drone; and are the maximum speed and maximum acceleration of the drone in each time slot, respectively.

[0114] The signal sent by the edge device is denoted as , where represents the wave velocity shaping design of the edge device at the th time slot, and represents the symbol of the edge device . Then, the signal received by the drone from the edge device is expressed as: ;

[0115] wherein, is zero-mean, with variance of additive white Gaussian noise.

[0116] Therefore, the achievable rate of the signal received by the UAV from the edge device is:

[0117] ;

[0118] The signal received by the eavesdropping device from the edge device is expressed as:

[0119] ;

[0120] where is the additive white Gaussian noise.

[0121] The achievable rate of the signal received by the eavesdropping device from the edge device is:

[0122] ;

[0123] The secure rate of the edge device is expressed as:

[0124] ; where .

[0125] The UAV senses and tracks the eavesdropping device in each time slot. First, the UAV sends a radar signal for sensing , where is the radar beam, is the radar symbol before precoding. Then, the radar signal reaches the eavesdropping device and is reflected back to the UAV, and is received by the receiving antenna equipped on the UAV. The radar echo signal received by the UAV in the th time slot is: ;

[0126] where is zero-mean, with variance of additive white Gaussian noise; is the UAV-eavesdropping device-UAV echo link, denoted as , where:

[0127] ;

[0128] where and are the wavelength and radar cross-section of the radar signal, respectively; represents the distance from the UAV to the eavesdropping device; and respectively represent the antenna array response vectors of the UAV receiving antenna and transmitting antenna.

[0129] To obtain satisfactory radar detection performance, the receive filter is adjusted to process the received echo signal, which can be expressed as: ;

[0130] Therefore, the output signal-to-noise ratio of the radar for target detection is: ;

[0131] The signal-to-noise ratio level calculated here will affect the subsequent positioning and tracking performance of the eavesdropping device.

[0132] In the offloading and computing stage of the edge device and the UAV, the edge device divides the data to be computed according to its own computing power. A part is computed on the processor equipped on the edge device, and the other part is sent to the UAV during the offloading stage and the remaining part is computed at the UAV side. Assume that the edge device is in the total amount of data to be computed in the th time slot is , the amount of data allocated for local computing is , and the remaining part is offloaded to the UAV for computing. The amount of offloaded data is , is the task allocation coefficient. In the local computing stage of the edge device, assume represents the number of CPU cycles required for user to process 1 bit of data, represents the CPU clock frequency of user . Then the time required for user to process bits of data and the energy consumed by the computing are:

[0133] ; ;

[0134] where is the effective capacitance coefficient of the edge device .

[0135] To avoid interference, in the offloading stage, frequency division multiple access is used for communication between each edge device and the UAV. The bandwidth is equally divided into sub-bandwidths. Then the time required for the edge device to offload to the UAV and the energy consumed can be expressed as:

[0136] ; ;

[0137] Among them, is the sub - bandwidth allocated to the edge device. Since the computing power of the UAV is much higher than that of the user, the number of bits related to the calculation result is very small, and the calculation time and task back - transmission time of the UAV can be ignored. In addition, when each edge device sends an offloading task, its transmission power is limited by the hardware, and their transmission power should satisfy: ... .

[0138] To reduce the total energy consumption of the edge users in the system, under the delay constraint, considering the computing offloading, the allocation of computing data tasks, the CPU clock frequency setting at the edge device, and the UAV flight trajectory, the optimization problem is modeled as follows:

[0139] ;

[0140] ;

[0141] Among them, is the security rate constraint to ensure that the information of the edge device is not eavesdropped by the eavesdropping device; is the security rate threshold; and are the local computing time constraint of the edge device and the time constraint of task offloading respectively; is the CPU clock frequency constraint of the edge device ; is the transmission power constraint of the edge device ; is the UAV trajectory constraint.

[0142] To solve this complex optimization problem, a UAV - edge computing framework based on the Extended Kalman Filter (EKF) is proposed to estimate the position of the target, and a deep reinforcement learning algorithm based on Proximal Policy Optimization (PPO) is proposed for jointly designing the transmit beamforming vector of the edge device, the radar beamforming vector of the UAV, the receive beamforming vector, the trajectory of the UAV, the speed of the UAV, and the CPU clock frequency setting and calculation task allocation coefficient of the edge device in the th time slot. In the above scenario, the position uncertainty of the mobile eavesdropping device brings great challenges to the secure transmission of the information of the power grid device. To ensure the secure transmission communication of the information of the power grid device, the power grid device must obtain the accurate and real - time position of the eavesdropping device before uploading the information data. For this problem, a method for edge computing and information security transmission of power grid devices based on the EKF - PPO algorithm is proposed. This method uses the EKF algorithm to estimate the position of the mobile eavesdropping device and the PPO algorithm to allocate the computing and communication resources of the power grid edge device.

[0143] See Figure 1 , a method for edge computing and information transmission of power grid equipment based on the EKF-PPO algorithm, including:

[0144] S1. Construct a state evolution model for the position and speed of a mobile eavesdropping device;

[0145] S2. Based on the echo signals received by the UAV, construct a radar measurement model;

[0146] S3. Predict the state of the mobile eavesdropping device and the MSE matrix based on the state evolution model;

[0147] S4. Calculate the channel between the UAV and the mobile eavesdropping device, and use the PPO algorithm to design the trajectory and speed of the UAV, the number of bits for local computing of the edge device and the number of bits uploaded to the UAV, the transmit beamforming vector of the edge device, and the frequency of local computing of the edge device;

[0148] S5. Calculate the Kalman gain using the radar measurement noise, and correct the state of the mobile eavesdropping device and the MSE matrix based on the Kalman gain and real-time radar measurement data.

[0149] Furthermore, based on the above Cartesian coordinate system, the position state and motion state of the mobile eavesdropping device can be naturally decomposed into two orthogonal directions, namely axis direction and axis direction. Assume that the mobile eavesdropping device moves at an approximately constant speed within the duration of a single time slot, and regard the change between two adjacent time slots as system noise. Therefore, the state evolution model for position and speed can be modeled as:

[0150] ; ;

[0151] ; ; ;

[0152] In the formula, is the time slot; is the state vector; is the linear state evolution matrix; is the state evolution noise, following a zero-mean Gaussian distribution; is the abscissa of the mobile eavesdropping device; is the ordinate of the mobile eavesdropping device; is the component of the speed of the mobile eavesdropping device in the axis; is the component of the speed of the mobile eavesdropping device in the axis; is the time slot length; is The state evolution error of is The state evolution error of is The state evolution error of is The state evolution error of is The covariance matrix of is The variance of is The variance of is The variance of is The variance of

[0153] Furthermore, based on the echo signal received by the UAV, the elevation angle and azimuth angle, delay and Doppler frequency shift of the mobile eavesdropping device relative to the UAV are estimated, and the radar measurement model is established as:

[0154] ; ;

[0155] ;

[0156] ;

[0157] ; ;

[0158] In the formula, is the measurement vector; is the measured value of the elevation angle of the UAV; is the measured value of the azimuth angle of the UAV; is the measured value of the Doppler frequency shift; is the measured value of the time delay; is the true value of the time delay; is the coordinate of the UAV; is the coordinate of the mobile eavesdropping device; is the speed of light; is the true value of the Doppler frequency shift; is the speed of the mobile eavesdropping device; is the wavelength of the signal; is the measurement noise, which follows a zero-mean Gaussian distribution; is the measurement error of the elevation angle; is the measurement error of the azimuth angle; is the measurement error of the time delay; is the measurement error of the Doppler frequency shift; is Covariance matrix; is the covariance matrix of the elevation angle and azimuth angle; is the variance of the time delay; is the variance of the Doppler shift;

[0159] Measurement vector and the state vector The relationship is expressed as:

[0160] ; ;

[0161] ;

[0162] In the formula, is the true value of the elevation angle; is the ordinate of the mobile eavesdropping device; is the ordinate of the UAV; is the abscissa of the mobile eavesdropping device; is the abscissa of the UAV; is the flight altitude of the UAV;

[0163] By introducing variables and , the covariance matrix of the elevation angle and azimuth angle is calculated using the Cramer-Rao lower bounds of the elevation angle and azimuth angle, expressed as:

[0164] ;

[0165] ; ;

[0166] In the formula, and are respectively and The Cramer-Rao lower bounds; is the number of received signal samples; is the number of receiving antennas; is the transmit signal-to-noise ratio; is the edge device -Large-scale fading of the sensing channel of the mobile eavesdropping device-UAV; is the transmit antenna steering vector from the UAV to the edge device ; is the transmit beamforming vector of the edge device ; is the signal propagation noise;

[0167] For the time delay and the Doppler shift The Cramer-Rao lower bound is calculated as follows:

[0168] ; ;

[0169] ; ;

[0170] ;

[0171] Wherein, is the bandwidth of the signal; is the received signal-to-noise ratio; is the slot length; is the sampling rate of the analog-to-digital converter; is the edge device - the sensing channel of the mobile eavesdropping device - the drone; is the received matched filter; is the receiving antenna steering vector from the drone to the edge device .

[0172] Furthermore, as Figure 3 shown, the position estimation algorithm of the mobile eavesdropping device based on EKF mainly includes the following three steps:

[0173] (1) Based on the state evolution model, predict the state of the mobile eavesdropping device at the th time slot:

[0174] ;

[0175] Approximate the non-linear radar measurement model as a linear measurement model by calculating the Jacobian matrix:

[0176] ;

[0177] Wherein, is the Jacobian matrix of; is the prior estimate of the state of the mobile eavesdropping device at the th time slot;

[0178] Estimate the prior estimate of the MSE matrix at the th time slot as follows:

[0179] ;

[0180] Wherein, is the MSE matrix at the th time slot; is the state evolution noise Covariance matrix.

[0181] (2) Real-time UAV trajectory and beamforming design:

[0182] After obtaining the position estimate of the mobile eavesdropping device in the th time slot, calculate the channel between the UAV and the mobile eavesdropping device:

[0183] ;

[0184] where is the channel between the UAV and the mobile eavesdropping device; is the channel gain at a relative distance of 1 m; is the distance between the UAV and the mobile eavesdropping device; is the receiving antenna steering vector from the UAV to the mobile eavesdropping device; is the transmitting antenna steering vector from the UAV to the mobile eavesdropping device; is the Hermitian conjugate of the vector.

[0185] Then, use the PPO algorithm to design the UAV's trajectory and speed, the number of bits calculated locally by the edge device and the number of bits uploaded to the UAV, the transmitting beamforming vector of the edge device, and the frequency of local calculation by the edge device.

[0186] (3) Mobile eavesdropping device state and MSE matrix correction:

[0187] Calculate the Kalman gain using the radar measurement noise: ;

[0188] where is the Kalman gain; is 's Jacobian matrix; is 's covariance matrix;

[0189] Use the Kalman gain and real-time radar measurement data to correct the state prediction to:

[0190] ;

[0191] where is the posterior estimate of the mobile eavesdropping device state; is the prior estimate of the mobile eavesdropping device state; is the measurement vector; represents and 's relationship;

[0192] The MSE matrix is updated to: ;

[0193] In the formula, is the identity matrix; is the prior estimate of the MSE matrix. The state of the mobile eavesdropping device and the MSE matrix in the -th time slot after correction are used to predict the state of the mobile eavesdropping device and the MSE matrix in the -th time slot.

[0194] Furthermore, the data interaction process between the grid edge device and the UAV is represented as a Markov decision process model. Among them, the UAV is naturally regarded as the agent, while the edge computing system and the signal propagation environment are regarded as the environment. The -th time slot is the -th step of the Markov decision model. The Markov decision process model is specifically:

[0195] The system state is various observations of the environment, including all channels in the edge computing system, the data volume of the edge device, and the positions of the UAV and the mobile eavesdropping device. The state at the -th step is:

[0196] ;

[0197] In the formula, is the state space of the Markov decision process model, The channels in are complex numbers. The real and imaginary parts must be separated before inputting into the neural network. Therefore, the input size is ; is the channel between the edge device and the UAV; is the channel between the edge device and the mobile eavesdropping device; is the sensing channel of the edge device -mobile eavesdropping device-UAV; is the data volume of the edge device ; is the position of the UAV; is the position of the mobile eavesdropping device;

[0198] The actions include the trajectory and speed of the UAV, the number of bits calculated locally by the edge device and the number of bits uploaded to the UAV, the transmit beamforming vector of the edge device, and the local computing frequency. The output layer of the Actor network uses tanh as the activation function of the output layer, and its output value is limited within the range of . The action at the -th step is:

[0199] ;

[0200] In the formula, is the action space of the Markov decision process model, and the output size is ; is the normalized value of the UAV transmission beamforming vector; is the normalized value of the CPU clock frequency of the edge device; is the ratio of the offloaded data volume to the total data volume; is the normalized value of the magnitude of the UAV's acceleration; is the normalized value of the direction of the UAV's acceleration;

[0201] To meet the actual constraint conditions, the speed of the UAV, the transmission beamforming vector of the edge device, and the frequency of local computing of the edge device are recovered through the following formulas:

[0202] ;

[0203] ;

[0204] ; ;

[0205] In the formula, is the component of the UAV speed on the axis; is the component of the UAV speed on the axis; is the maximum speed of the UAV in each time slot; is the normalized value of the magnitude of the UAV's acceleration; is the time slot length; is the maximum acceleration of the UAV in each time slot; is the th time slot of the UAV speed; is the normalized value of the direction of the UAV's acceleration; is the edge device in the th time slot of the wave speed shaping design; is the maximum transmission power of the UAV; is the edge device in the th time slot of the normalized value of the wave speed shaping design; is the number of transmission antennas of the UAV; is the edge device 's CPU clock frequency; is the edge device 's normalized value of the CPU clock frequency; is the maximum CPU clock frequency of the edge device; is the minimum CPU clock frequency of the edge device;

[0206] Since the deep reinforcement learning algorithm maximizes the total reward, the reward for each step is:

[0207] ;

[0208] ;

[0209] ;

[0210] ;

[0211] ;

[0212] ;

[0213] ;

[0214] where is the function obtained by executing the th action; is the number of edge devices; is the energy consumed by the edge device to process data; is the energy consumed by the edge device to upload data; is the data processing time penalty function; is the penalty function for data upload time; , are the weights of the penalty functions; is the security rate of the edge device ; is the data processing time; is the data upload time; is the threshold of data processing time and upload time; is the effective capacitance coefficient; is the CPU frequency of the edge device ; is the ratio of the amount of offloaded data of the edge device to the total amount of data; is the number of bits of the total amount of data; is the number of CPU cycles required for the edge device to process 1 bit of data; is the bandwidth of the signal.

[0215] Furthermore, PPO is a model-free and policy-based algorithm, and its architecture is as shown in Figure 4 and Figure 5 where Figure 4Represents the interaction process between the agent and the environment, Figure 5 represents the update process of the Actor network and the Critic network. The agent includes an Actor network and a Critic network, which obtains state information from the environment and inputs the state into the Actor network and the Critic network, and outputs actions and state values respectively:

[0216] The Actor network adopts a 4-layer fully connected network. As Figure 6 shown, the state is used as the input, and the mean vector and standard deviation vector of the action are output. The action policy following a Gaussian distribution is:

[0217] ;

[0218] In the formula, is the new action policy; is the action at the th step, obtained by sampling from the new action policy ; is the state; are the parameters of the Actor network;

[0219] To prevent the action policy from falling into sub-optimal actions, an advantage function is introduced to measure the quality of the action. Therefore, the loss function of the actor is designed as:

[0220] ;

[0221] ; ;

[0222] In the formula, is the probability ratio of the new action policy to the old action policy; is the old action policy; is the advantage function; is the action value function; is the state value.

[0223] Repeat the above steps until a complete round of the process is carried out (until the end of the th time slot). During this process, the agent continuously interacts with the environment to generate experiences, and stores these experiences in the experience pool for policy optimization. The experience data can be expressed as:

[0224] ;

[0225] Empty the experience pool until all samples in the experience pool have been used for model training.

[0226] In practical applications, it is difficult to accurately calculate the advantage function. , and the present invention uses the Generalized Advantage Estimation (GAE) method to estimate it:

[0227] ; ;

[0228] In the formula, is the temporal difference; is the discount factor; is the maximum number of steps;

[0229] If there is a significant difference between the new action policy and the old action policy , the extreme volatility of the action policy update may lead to a performance decline during the maximization of the objective function. To solve this problem, a clipping method is used to process the loss function , which is rewritten as:

[0230] ;

[0231] ;

[0232] In the formula, is the processed loss function; is the clipping factor; is the clipping function;

[0233] The Critic network uses a 4-layer fully connected network. As Figure 7 shown, the state is used as the input, and the state value is output. The goal of the Critic network is to minimize the gap between the actual value and . Therefore, the objective function for updating the Critic network is:

[0234] ;

[0235] ;

[0236] In the formula, is the loss function of the Critic network; is the cumulative discounted reward; is the discount factor to the power; is the function obtained by performing the action at the th step; is the time period.

[0237] Based on the loss functions of the Actor network and the Critic network, the Adam optimizer is used to update the network parameters, and the update formulas are as follows:

[0238] ; ;

[0239] ; ;

[0240] In the formula, and are the network parameters of the Actor network and the Critic network, is the update step size.

[0241] Repeat the above steps until the loss function converges, and the PPO model training is completed.

[0242] The present invention has the following advantages: (1) It improves the edge computing ability of power grid equipment. By deploying edge computing servers on drones, the data of power grid equipment can be processed in real time. Through edge computing technology, drones can quickly analyze and make decisions on the status data of power grid equipment without relying on remote servers, thus reducing the data transmission delay and enhancing the real-time response ability. Edge computing enables drones to independently complete local data processing and optimization decisions based on information such as the operating status of power grid equipment and environmental changes, improving the system's adaptability, especially in remote areas and complex environments, and providing more efficient and reliable power grid monitoring. (2) It enhances the flexibility and coverage ability of drones. Drones have the advantages of rapid deployment and flexible flight, and can efficiently cover areas where power grid equipment is widely distributed and the terrain is complex. Drones can not only automatically plan flight paths but also adjust flight strategies in real time according to the actual situation of power grid equipment and environmental changes. Their flexibility enables drones to provide communication relays in areas where conventional base stations cannot cover, such as remote areas and mountainous areas, ensuring remote monitoring and data transmission of power grid equipment. At the same time, drones can autonomously avoid threats from interference and eavesdroppers, enhancing the reliability and security of communication. (3) The dynamic optimization and decision-making ability of the EKF-PPO algorithm. By combining the EKF and PPO algorithms, this technical solution realizes the dynamic optimization of the drone's flight trajectory, communication beam, and data offloading decision. The EKF algorithm estimates the position of mobile eavesdropping devices in real time to ensure that the offloaded data can avoid being exposed to eavesdropping devices; at the same time, the PPO algorithm optimizes the drone's flight trajectory, communication beam, and data offloading decision to ensure the security and efficiency of signal transmission.

[0243] See Figure 8, the present invention also provides a power grid device edge computing and information transmission system based on the EKF-PPO algorithm. This system is applied to the power grid device edge computing and information transmission method based on the EKF-PPO algorithm described above. The system includes: a state evolution model construction module for constructing a state evolution model of the position and speed of a mobile eavesdropping device; a radar measurement model construction module for constructing a radar measurement model based on the echo signals received by the unmanned aerial vehicle; a mobile eavesdropping device state prediction module for predicting the state of the mobile eavesdropping device and the MSE matrix based on the state evolution model; a PPO algorithm module for calculating the channel between the unmanned aerial vehicle and the mobile eavesdropping device, and using the PPO algorithm to design the trajectory and speed of the unmanned aerial vehicle, the number of bits for local computing of the edge device and the number of bits uploaded to the unmanned aerial vehicle, the transmit beamforming vector of the edge device, and the frequency of local computing of the edge device; a mobile eavesdropping device position correction module for calculating the Kalman gain using the radar measurement noise, and correcting the state of the mobile eavesdropping device and the MSE matrix based on the Kalman gain and real-time radar measurement data.

[0244] See Figure 9 , the present invention also provides a power grid device edge computing and information transmission device based on the EKF-PPO algorithm, including a memory and a processor; the memory is used for storing computer program code and transmitting the computer program code to the processor; the processor is used for executing the power grid device edge computing and information transmission method based on the EKF-PPO algorithm described above according to the instructions in the computer program code. The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the power grid device edge computing and information transmission method based on the EKF-PPO algorithm described above.

Claims

1. A method for edge computing and information transmission of power grid equipment based on the EKF-PPO algorithm, characterized in that Including: Construct a state evolution model for the position and velocity of a mobile eavesdropping device; Based on the echo signals received by the UAV, construct a radar measurement model; Predict the state of the mobile eavesdropping device and the MSE matrix based on the state evolution model; Based on the state evolution model, predict the state of the mobile eavesdropping device at the th time slot: ； Approximate the non-linear radar measurement model as a linear measurement model by calculating the Jacobian matrix: ； In the formula, is the Jacobian matrix of; is the prior estimate of the state of the mobile eavesdropping device at the th time slot; The prior estimate of the MSE matrix for the th time slot is as follows: ； In the formula, is the MSE matrix of the th time slot; is the covariance matrix of the state evolution noise . Calculate the channel between the UAV and the mobile eavesdropping device, and use the PPO algorithm to design the trajectory and velocity of the UAV, the number of bits calculated locally by the edge device and the number of bits uploaded to the UAV, the transmit beamforming vector of the edge device, and the frequency of local calculation of the edge device; The channel between the UAV and the mobile eavesdropping device is: ； In the formula, is the channel between the UAV and the mobile eavesdropping device; is the channel gain at a relative distance of 1 m; is the distance between the UAV and the mobile eavesdropping device; is the receiving antenna steering vector pointing from the UAV to the mobile eavesdropping device; is the transmitting antenna steering vector pointing from the UAV to the mobile eavesdropping device; is the Hermitian conjugate of the vector; Calculate the Kalman gain using the radar measurement noise, and correct the state of the mobile eavesdropping device and the MSE matrix based on the Kalman gain and real-time radar measurement data.

2. The method for edge computing and information transmission of power grid equipment based on the EKF-PPO algorithm according to claim 1, characterized in that The state evolution model of the mobile eavesdropping device is: ； ； ； ； ； Wherein, is a time slot; is a state vector; is a linear state evolution matrix; is state evolution noise; is the abscissa of the mobile eavesdropping device; is the ordinate of the mobile eavesdropping device; is the component of the velocity of the mobile eavesdropping device on the axis; is the component of the velocity of the mobile eavesdropping device on the axis; is the time slot length; is the state evolution error of; is the state evolution error of; is the state evolution error of; is the state evolution error of; is the covariance matrix of; is the variance of; is the variance of; is the variance of; is the variance of.

3. A method for edge computing and information transmission of power grid equipment based on the EKF-PPO algorithm according to claim 1, characterized in that The radar measurement model is: ； ； ； ； ； ； In the formula, is the measurement vector; is the measured value of the elevation angle of the UAV; is the measured value of the azimuth angle of the UAV; is the measured value of the Doppler frequency shift; is the measured value of the time delay; is the true value of the time delay; is the coordinate of the UAV; is the coordinate of the mobile eavesdropping device; is the speed of light; is the true value of the Doppler frequency shift; is the speed of the mobile eavesdropping device; is the wavelength of the signal; is the measurement noise; is the measurement error of the elevation angle; is the measurement error of the azimuth angle; is the measurement error of the time delay; is the measurement error of the Doppler frequency shift; is the covariance matrix of; is the covariance matrix of the elevation angle and the azimuth angle; is the variance of the time delay; is the variance of the Doppler frequency shift; Measurement vector and the state vector is related as expressed as: ； ； ； Wherein, is the true value of the elevation angle; is the ordinate of the mobile eavesdropping device; is the ordinate of the UAV; is the abscissa of the mobile eavesdropping device; is the abscissa of the UAV; is the flight altitude of the UAV; By introducing variables and , the covariance matrix of the elevation angle and azimuth angle is calculated using the Cramer-Rao lower bounds of the elevation angle and azimuth angle, expressed as: ； ； ； Wherein, and are respectively and 's Cramer-Rao lower bounds; is the number of received signal samples; is the number of receiving antennas; is the transmit signal-to-noise ratio; is the large-scale fading of the sensing channel of the edge device -mobile eavesdropping device-drone; is the transmit antenna steering vector from the drone to the edge device ; is the transmit beamforming vector of the edge device ; is the signal propagation noise; For time delay and Doppler frequency shift the Cramer-Rao lower bound is calculated as follows: ； ； ； ； ； wherein, is the bandwidth of the signal; is the received signal-to-noise ratio; is the time slot length; is the sampling rate of the analog-to-digital converter; is the edge device -mobile eavesdropping device-drone sensing channel; is the received matching filter; is from the drone to the edge device received antenna steering vector.

4. A method for edge computing and information transmission of power grid devices based on the EKF-PPO algorithm according to claim 1, characterized in that Calculate the Kalman gain using the radar measurement noise: ； In the formula, is the Kalman gain; is the Jacobian matrix of is the covariance matrix of Using the Kalman gain and real-time radar measurement data, correct the state prediction to: ； In the formula, is the posterior estimate of the state of the mobile eavesdropping device; is the prior estimate of the state of the mobile eavesdropping device; is the measurement vector; denotes the relationship between and The MSE matrix is updated to: ； In the formula, is the identity matrix; is the prior estimate of the MSE matrix.

5. A method for edge computing and information transmission of power grid equipment based on the EKF-PPO algorithm according to claim 1, characterized in that, Represent the data interaction process between the power grid edge device and the UAV as the following Markov decision process model: Step The status is: ； wherein, is the state space of the Markov decision process model; is the channel between the edge device and the UAV; is the channel between the edge device and the mobile eavesdropping device; is the sensing channel of the edge device -mobile eavesdropping device-UAV; is the data volume of the edge device ; is the position of the UAV; is the position of the mobile eavesdropping device; Step The action of is: ； In the formula, is the action space of the Markov decision process model; is the normalized value of the UAV transmission beamforming vector; is the normalized value of the CPU clock frequency of the edge device; is the ratio of the offloaded data volume to the total data volume; is the normalized value of the magnitude of the UAV's acceleration; is the normalized value of the direction of the UAV's acceleration; The velocity of the UAV, the transmit beamforming vector of the edge device, and the frequency of local calculation of the edge device are recovered by the following formulas: ； ； ； ； wherein, is the component of the UAV speed in the axis; is the component of the UAV speed in the axis; is the maximum speed of the UAV in each time slot; is the normalized value of the magnitude of the UAV acceleration; is the time slot length; is the maximum acceleration of the UAV in each time slot; is the th UAV speed in the time slot; is the normalized value of the direction of the UAV acceleration; is the edge device in the th wave speed shaping design in the time slot; is the maximum transmission power of the UAV; is the edge device in the th normalized value of the wave speed shaping design in the time slot; is the number of transmitting antennas of the UAV; is the CPU clock frequency of the edge device ; is the normalized value of the CPU clock frequency of the edge device ; is the maximum CPU clock frequency of the edge device; is the minimum CPU clock frequency of the edge device; The reward is: ； ； ； ； ； ； ； Wherein, is the function obtained by performing the step action; is the number of edge devices; is the energy consumed by the edge device to process data; is the energy consumed by the edge device to upload data; is the data processing time penalty function; is the penalty function for data upload time; , are the weights of the penalty function; is the security rate of the edge device; is the data processing time; is the data upload time; is the threshold of data processing time and upload time; is the effective capacitance coefficient; is the CPU frequency of the edge device; is the ratio of the amount of offloaded data to the total amount of data of the edge device; is the number of bits of the total data volume; is the number of CPU cycles required for the edge device to process 1 bit of data; is the bandwidth of the signal.

6. A method for edge computing and information transmission of power grid equipment based on the EKF-PPO algorithm according to claim 5, characterized in that, Input the state into the Actor network and the Critic network, and output the action and the state value respectively: The Actor network uses a 4-layer fully connected network, with the state as the input, and outputs the mean vector of the actions and the standard deviation vector . The action policy is as follows: ； In the formula, is the new action strategy; is the action of the th step; is the state; are the parameters of the Actor network; The loss function of the actor is: ； ； ； Wherein, is the probability ratio of the new action policy to the old action policy; is the old action policy; is the advantage function; is the action value function; is the state value; Estimate using the generalized advantage estimation method: ； ； In the formula, is the temporal difference; is the discount factor; is the maximum number of steps; Using a clipping method to process the loss function It is as follows: ； ； In the formula, is the processed loss function; is the clipping factor; is the clipping function; The Critic network uses a 4-layer fully connected network, with the state as the input, and outputs the state value , and the objective function of the Critic network is updated as follows: ； ； In the formula, is the loss function of the Critic network; is the cumulative discounted reward; is for executing the function obtained by the action in step is the time period.

7. A grid device edge computing and information transmission system based on the EKF-PPO algorithm, characterized in that, The system is applied to the method described in any one of claims 1-6, and the system includes: A state evolution model construction module for constructing a state evolution model for the position and velocity of a mobile eavesdropping device; A radar measurement model construction module for constructing a radar measurement model based on the echo signals received by the UAV; A mobile eavesdropping device state prediction module for predicting the state of the mobile eavesdropping device and the MSE matrix based on the state evolution model; A PPO algorithm module for calculating the channel between the UAV and the mobile eavesdropping device, and using the PPO algorithm to design the trajectory and velocity of the UAV, the number of bits calculated locally by the edge device and the number of bits uploaded to the UAV, the transmit beamforming vector of the edge device, and the frequency of local calculation of the edge device; A mobile eavesdropping device position correction module for calculating the Kalman gain using the radar measurement noise, and correcting the state of the mobile eavesdropping device and the MSE matrix based on the Kalman gain and real-time radar measurement data.

8. A power grid device edge computing and information transmission device based on the EKF-PPO algorithm, characterized in that Including a memory and a processor; The memory is used to store computer program code and transmit the computer program code to the processor; The processor is used to execute the method described in any one of claims 1 to 6 according to the instructions in the computer program code.

Images