Rate optimization method of unmanned aerial vehicle information acquisition system based on sensing integration

Abstract

The invention relates to the technical field of wireless communication, and discloses an unmanned aerial vehicle information acquisition system rate optimization method based on communication and inductance integration, and the method comprises the steps: determining a baseband equivalent channel from an unmanned aerial vehicle to ground Internet of Things equipment; determining a data signal received by the unmanned aerial vehicle through a receiving antenna based on the baseband equivalent channel; determining the data rate of the ith ground Internet of Things equipment collected by the mth unmanned aerial vehicle; establishing an optimization problem based on the constraint condition and the data rate; and solving the optimization problem through a multi-agent reinforcement learning model, and determining the maximum total rate of collecting information from the I ground Internet of Things devices by the M unmanned aerial vehicles. According to the invention, while the sensing precision requirement, unmanned aerial vehicle track limitation, power limitation and distance limitation between unmanned aerial vehicles are met, joint optimization is carried out on the track, transmitted beam forming and sensing signals of the unmanned aerial vehicles, so that the purposes of high-speed signal transmission and efficient information collection are achieved, and the total transmission rate of an unmanned aerial vehicle information collection system is maximized.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a rate optimization method for an unmanned aerial vehicle information collection system based on synaesthesia integration. Background Art

[0002] The Integrated Sensing and Communication (ISAC) system is an emerging technology that integrates wireless communication and environmental perception functions on the same platform. It aims to achieve efficient communication and accurate environmental perception by sharing wireless spectrum and hardware resources. The ISAC system has broad application prospects in many fields such as intelligent transportation, unmanned driving, smart cities and industrial Internet of Things. The ISAC system can realize wireless communication and environmental perception at the same time under the same spectrum resources through spectrum sharing. The network can be used for data transmission and radar detection at the same time, which significantly improves spectrum utilization. Through hardware sharing, such as antenna arrays and RF front-ends, millimeter-wave radars and communication base stations share antennas, which reduces the need for independent communication and perception equipment, reduces hardware costs and system complexity. Through signal processing integration and information fusion, communication and perception performance are collaboratively optimized to improve the overall efficiency and reliability of the system.

[0003] With the development of drone technology, ISAC systems have begun to be widely used in the drone field. By integrating perception and communication functions, drones can provide ground users with wireless coverage, reconnaissance, plant protection operations and other tasks in a wide area environment, allowing drone swarms to autonomously plan flight paths and optimize trajectories in unknown environments, improving flight safety and work efficiency. Driven by the on-demand deployment of drones and strong line-of-sight links, drones have become an aerial ISAC platform, providing more controllable and balanced integrated services based on the requirements of sensing frequency and communication quality. Due to the complexity of spectrum resource sharing and signal scattering, achieving a good balance between the two potentially conflicting design goals of high-quality communication services and high-time perception needs is the key to ensuring interawareness performance. The design of beamforming is one of the core issues of ISAC, and the technical focus is on how to design a beam array matrix that can meet both communication and perception needs.

[0004] To solve this problem, array signal processing algorithms have been developed, such as the minimum mean square error (MMSE) algorithm and adaptive filtering technology, which monitor the environment and target information in real time and dynamically adjust the beam direction and shape to optimize signal quality and perception performance; Multiple-Input Multiple-Output (MIMO) technology uses multiple antennas to simultaneously transmit and receive signals, improving the spatial resolution and signal strength of the system to achieve efficient beamforming; Massive MIMO uses a large number of antenna units to form an extremely narrow beam, significantly improving the system's angular resolution and perception accuracy, and achieving efficient signal transmission and reception through precise antenna arrangement and advanced signal processing algorithms.

[0005] With the development of artificial intelligence in recent years, the application of reinforcement learning has attracted more and more attention. The core concept of reinforcement learning is the interaction between the intelligent agent and the environment. Through a series of behaviors of the intelligent agent, feedback after the behavior is made, and continuous iteration, the optimal result of the problem is finally obtained. Reinforcement learning is applied in the field of wireless communications. Because the environment is changing, complex and random, traditional algorithms are difficult to adapt to these characteristics and cannot make corresponding optimal decisions in a rapidly changing environment. In reinforcement learning, the intelligent agent can provide feedback based on the real-time status of the environment to make the best decision in different environments. At the same time, reinforcement learning improves the efficiency of network management, and its ability to make autonomous decisions can effectively utilize existing wireless and computing resources.

[0006] Due to its advantages in policy optimization in complex environments, Deep Reinforcement Learning (DRL) technology has gradually become an effective tool for solving beamforming problems in ISAC systems. Beamforming is a technology that improves communication and perception performance by controlling the transmission and reception directions of antenna arrays. Traditional beamforming methods, such as optimal beam design based on precoding, usually require accurate channel state information (CSI) and complex calculations. In actual environments, CSI is often difficult to obtain accurately, and the environment is dynamically changing, requiring real-time adjustment of the beam direction. In deep reinforcement learning applications, for modeling the environment, the state of the agent may include channel state information, interference conditions, target location, and motion state, etc. The action space is defined as the beam direction of the antenna array, power allocation strategy, or multi-beam combination strategy, etc. High-dimensional features are extracted through deep neural networks (such as convolutional neural networks, CNN) to obtain the optimal configuration of the system.

[0007] Currently, two specific methods are applied in solving the problem of drone trajectory optimization using deep reinforcement learning: Deep Q Network (DQN) and Double DQN (DDQN) methods, which are suitable for discrete action space, evaluate the value of each action through the Q network, and select the optimal action. First, collect environmental state data, communication channel information data, and target perception data, and preprocess them to meet the input requirements of the DRL model. Then train the model, select the appropriate DRL algorithm, build a deep neural network model, and define the state, action, and reward functions. The model is trained in a simulated environment, and the strategy is updated through a large amount of interaction data to optimize the drone trajectory. As a reinforcement learning algorithm based on deep learning, the DQN algorithm performs well in the discrete action space problem of a single agent. However, in the scenario of multi-drone cooperation, especially when the agent is unknown to the decision made by other agents, a good solution cannot be obtained. Multi-agent reinforcement learning (MARL) and multi-agent deep reinforcement learning (MADRL) provide solutions to the problem of multi-agent joint training based on single-agent reinforcement learning. Their algorithms can still make optimal decisions when a UAV is unaware of the decisions of other UAVs, effectively solving problems such as multi-UAV joint execution of tasks and resource scheduling, and achieving superior system performance.

[0008] Previous developments on the ISAC system have focused on improving the performance of both communication and perception, ignoring the actual asymmetric sensing and communication requirements, and the exploration of the trajectory optimization problem of drones under the ISAC model is not sufficient. At the same time, there are still limitations in the research on drone trajectory optimization. Most studies only consider the trajectory optimization of single or multiple drones with power as a constraint, ignoring the problems existing in actual ISAC scenarios, such as the impact of the location of drones as communication relay stations on communication quality, the relationship between drone perception accuracy and its trajectory and energy consumption, etc. Therefore, it is necessary to jointly develop drone trajectory optimization with various factors under the model. Summary of the invention

[0009] In view of this, the present invention provides a rate optimization method for a drone information collection system based on synaesthesia integration, which solves the problem that the prior art cannot ensure the communication rate between drones and IoT devices while ensuring perception accuracy, thereby affecting the overall path optimization of the drone swarm.

[0010] In a first aspect, the present invention provides a method for optimizing the rate of a UAV information collection system based on synaesthesia integration, the method comprising:

[0011] Step S1, determining the baseband equivalent channel from the drone to the ground IoT device; wherein step S1 includes:

[0012] Step S11, determine the channel gain β from the drone to the user m,i , In the formula, represents the two-dimensional plane position of the UAV m at height G, Represents the two-dimensional coordinate information of the i-th IoT device, β0 represents the channel gain at 1 meter, n is the time slot, and the total number of time slots is N;

[0013] Step S12: Based on the channel gain, determine the baseband equivalent channel h from the mth UAV to the i-th ground IoT device m,i ,in, Where, d m,i represents the distance between the mth UAV and the ith ground IoT device. The superscript H represents the conjugate transpose. is the response vector of the UAV m to the user’s transmitted signal, where b is the antenna half-wavelength margin, λ is the carrier wavelength, and N t Represents the total number of transmitting antennas of the drone. The number of transmitting antennas is the same as that of receiving antennas. m,i is the elevation angle of the geographic path between the mth UAV and the i-th ground IoT device,

[0014] Step S2, based on the baseband equivalent channel, determine the data signal received by the drone through the receiving antenna

[0015] In the formula, α m,i is the radar cross-sectional area of ​​the UAV m, which satisfies the normal distribution. is the perception signal transmitted by UAV m in time slot n, represents the perception signal emitted by UAV m, and represents the transmit beam precoding vector of the perception signal, and The data sent by the ground IoT device i to the drone m. represents the receive beam coding vector of the data signal, and represents additive Gaussian white noise, satisfying The superscript T indicates transposition;

[0016] Step S3, determine the data rate R of the mth UAV collecting the i-th ground IoT device m,i [n], R m,i [n] = log2(1 + γm,i [n]), where

[0017] Step S4, establishing an optimization problem based on the constraints and the data rate;

[0018] The optimization problem is: In the formula, is the covariance matrix of the perception signal, M is the total number of drones, and I is the total number of ground IoT devices;

[0019] The constraints include:

[0020] In the formula, C1 represents the trajectory constraint for all drones, C2 represents the power constraint for drones, C3 represents the perception accuracy requirement for drones, C4 represents the minimum distance constraint between drones to avoid collision, S represents the radius of the entire coverage area, P represents the power upper limit, Γ represents the perception accuracy limit, and L min It represents the minimum separation distance between drones, and k represents drones other than drone m;

[0021] Step S5, solving the optimization problem through a multi-agent reinforcement learning model to determine the maximum total rate at which M drones collect information from I ground IoT device.

[0022] The present invention considers the ISAC system scenario including multiple drones and multiple ground IoT devices, and establishes an optimization problem for the constructed drone information collection system. While satisfying the perception accuracy, drone trajectory restrictions, power restrictions and distance between drones, the drone trajectory, transmission beamforming and transmission symbols are jointly optimized to achieve high-speed signal transmission and efficient information collection, so as to maximize the total transmission rate of the drone information collection system. In order to better reflect the mutual assistance of multiple agents in actual scenarios and avoid interference between agents, the present invention also combines a multi-agent deep reinforcement learning method to obtain the optimal agent action setting for the transmission rate of the drone information collection system, thereby realizing the overall path optimization of the drone group.

[0023] In an optional implementation, the CRB(Φ) estimated by the parameter Φ is determined by the following steps:

[0024] Determine the parameters that need to be sensed in,

[0025] The echo signal received by the drone is vectorized, and the data signal transmitted by the ground IoT device to the drone is regarded as noise, that is,

[0026] Determine the data signal received by the drone:

[0027] Will Convert to: Y = αHX + V, where

[0028] Vectorize Y = αHX + V to determine η=vec{αHX}, in, The Fisher information matrix F IM The element in the mth row and ith column is represented as: Among them, ξ m and i represents the mth and ith elements of ξ;

[0029] Divide the Fisher information matrix into a 2x2 matrix F IM ,in, In the formula, represents the matrix for finding the second partial derivative of the parameter Φ, Indicates the parameters Find the matrix of the second partial derivative, Represents the parameters Φ and Find the matrix of partial derivatives respectively;

[0030] According to the relationship between the Fisher information matrix and the CRB matrix, the CRB estimated for the parameter Φ is:

[0031] In an optional implementation, the optimization problem is solved by a multi-agent reinforcement learning model to determine the maximum total rate at which M drones collect information from I ground IoT device, including:

[0032] Determine that the M drones are M agents, and determine the state set, observation set, and action set corresponding to each agent;

[0033] Determine the strategy corresponding to each agent based on the state set, observation set, and action set;

[0034] Based on the optimization problem, determine the reward function, where the reward function is: In the formula, β represents the penalty factor, p1, p2, p3, and p4 are all penalty items, where:

[0035]

[0036] In the formula, F(f(X)) represents the discriminant value after being judged by the discriminant function f(X), where:

[0037]

[0038] Obtain a pre-established policy decision model, which includes an Actor-Critic network;

[0039] Based on the reward function and the strategy decision model, a decision is made for each strategy to determine the optimal strategy so that the total rate at which M drones collect information from I ground IoT device is maximized.

[0040] In an optional embodiment, the Actor network in the Actor-Critic network is used to make an action decision for each observation according to the policy, and the Critic network is used to evaluate the action selected by the Actor network using a state-value function;

[0041] Among them, the state-value function is:

[0042]

[0043] In the formula, γ represents the discount factor, τ represents the power parameter of the discount factor, and r m represents the reward of agent m, represents the input state of agent m, a m Indicates the action corresponding to the input state.

[0044] In an optional implementation, the Actor-Critic network determines the optimal strategy by adjusting the parameters of the Actor network and the Critic network; wherein the Actor-Critic network is composed of an evaluation network and a target network, and the evaluation network includes the parameters of the Actor network And the parameters of the Critic network The target network includes the parameters of the Actor network And the parameters of the Critic network

[0045] The evaluation network parameters of the critic network are adjusted by reducing the loss, and the lth state transfer The loss function is: Among them, H′(·) is the state value function of the target network. Under continuous and differentiable conditions, the Critic network is adjusted according to the gradient of the loss function;

[0046] The evaluation network parameters of the Actor network are updated by maximizing the policy objective function, which is: In the formula, χ m (·) is the evaluation network function of the Actor network;

[0047] The target network parameters of the Critic network are updated using the first function, which is: In the formula, κ is a positive constant;

[0048] The target network parameters of the Actor network are updated using the second function, which is:

[0049] In an optional implementation, the Actor-Critic network adjusts the parameters of the Actor network and the Critic network, including:

[0050] Initialize the evaluation network parameters and target network parameters of the Actor network and Critic network corresponding to each agent;

[0051] Initialize the experience pool size of each agent;

[0052] Determine time slot n and agent m;

[0053] Observe the environment of agent m and obtain state s m [n];

[0054] According to the status m [n], agent m chooses to perform action a m [n];

[0055] After agent m performs an action, it obtains the next time slot state s m [n+1], until all actions a of agent m m [n] The execution is completed, and all states, actions, and rewards are stored in the experience pool;

[0056] Select experience samples from the experience pool;

[0057] Based on the empirical samples, the evaluation network parameters of the Critic network are updated by minimizing the loss function, the evaluation network parameters of the Actor network are updated by maximizing the strategy objective function, the target network parameters of the Critic network are updated by the first function, and the target network parameters of the Actor network are updated by the second function.

[0058] In a second aspect, the present invention provides a rate optimization device for a UAV information collection system based on synaesthesia integration, the device comprising:

[0059] The baseband equivalent channel determination module is used to determine the baseband equivalent channel from the drone to the ground IoT device; it includes: a module for determining the channel gain β from the drone to the user m,i , In the formula, represents the two-dimensional plane position of the UAV m at height G, Represents the two-dimensional coordinate information of the i-th IoT device, β0 represents the channel gain at 1 meter, n is the time slot, and the total number of time slots is N; it is also used to determine the baseband equivalent channel h from the mth drone to the i-th ground IoT device based on the channel gain m,i ,in, Where, d m,i represents the distance between the mth UAV and the ith ground IoT device. The superscript H represents the conjugate transpose. is the response vector of the transmitted signal of UAV m to user i, where b is the half-wavelength margin of the antenna, λ is the carrier wavelength, and N t Represents the total number of transmitting antennas of the drone. The number of transmitting antennas is the same as that of receiving antennas. m,i is the elevation angle of the geographic path between the mth UAV and the i-th ground IoT device,

[0060] The data signal determination module is used to determine the data signal received by the drone through the receiving antenna based on the baseband equivalent channel In the formula, α m,i is the radar cross-sectional area of ​​the UAV m, which satisfies the normal distribution. is the perception signal transmitted by UAV m in time slot n, represents the perception signal emitted by UAV m, and represents the transmit beam precoding vector of the perception signal, and The data sent by the ground IoT device i to the drone m. represents the receive beam coding vector of the data signal, and z m,i [n] represents additive Gaussian white noise, satisfying The superscript T indicates transposition;

[0061] The rate determination module is used to determine the data rate R of the mth UAV collecting the i-th ground IoT device m,i [n], R m,i [n] = log2(1 + γ m,i [n]), where

[0062] A module is established to establish an optimization problem based on constraints and data rates; wherein the optimization problem is: In the formula, is the covariance matrix of the perception signal, M is the total number of drones, and I is the total number of ground IoT devices;

[0063] The constraints include:

[0064] In the formula, C1 represents the trajectory constraint for all drones, C2 represents the power constraint for drones and ground IoT devices, C3 represents the perception accuracy requirement for drones, C4 represents the minimum distance constraint between drones to avoid collision, S represents the radius of the entire coverage area, P represents the power upper limit, Γ represents the perception accuracy limit, and L min It represents the minimum separation distance between drones, and k represents drones other than drone m;

[0065] The optimization module is used to solve the optimization problem through a multi-agent reinforcement learning model to determine the maximum total rate at which M drones collect information from I ground IoT device.

[0066] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the rate optimization method of a drone information collection system based on synaesthesia integration of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0067] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions, and the computer instructions are used to enable a computer to execute the rate optimization method of a drone information collection system based on synaesthesia integration according to the above-mentioned first aspect or any corresponding embodiment thereof.

[0068] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, which are used to enable a computer to execute the rate optimization method of a drone information collection system based on synaesthesia integration according to the above-mentioned first aspect or any corresponding embodiment thereof.

[0069] It should be noted that the rate optimization device for the unmanned aerial vehicle information collection system based on synesthesia integration, the computer device and the computer-readable storage medium provided by the present invention correspond to the rate optimization method for the unmanned aerial vehicle information collection system based on synesthesia integration. Therefore, for the beneficial effects of the rate optimization device for the unmanned aerial vehicle information collection system based on synesthesia integration, the computer device and the computer-readable storage medium, please refer to the description of the corresponding beneficial effects of the rate optimization method for the unmanned aerial vehicle information collection system based on synesthesia integration above, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 1 is a flow chart of a method for optimizing the rate of a UAV information collection system based on synaesthesia integration according to an embodiment of the present invention;

[0072] Figure 2 is a schematic diagram of a drone information collection system according to an embodiment of the present invention;

[0073] Figure 3 is a structural block diagram of a rate optimization device for a UAV information collection system based on synaesthesia integration according to an embodiment of the present invention;

[0074] Figure 4 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0076] In view of this, according to an embodiment of the present invention, an embodiment of a method for optimizing the rate of a drone information collection system based on synaesthesia integration is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0077] In this embodiment, a method for optimizing the rate of a drone information collection system based on synaesthesia integration is provided, which can be executed by satellites, drones, Internet of Things devices, etc. Figure 1 is a flow chart of a rate optimization method for a UAV information collection system based on synaesthesia integration according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0078] Step S1, determine the baseband equivalent channel from the drone to the ground IoT device. Figure 2As shown, a drone information collection system supporting ISAC is provided, wherein the drone information collection system includes a drone group and multiple ground IoT devices. A connection is established between the drone and the IoT device to sense the location of the IoT device and receive data uploaded by the IoT device.

[0079] In this embodiment, the drone information collection system deploys multiple drones supporting ISAC functions to work in N time slots. The total number of time slots is N. The drone is equipped with N t A uniform linear array of transmitting antennas is used to transmit sensing signals to sense ground IoT devices, N t Indicates the total number of transmitting antennas of the drone, for example, if there are 10 transmitting antennas, N r receiving antennas for receiving signals, for example, 10 receiving antennas, where N t >1,N r >1,N t =N r ,The sensing signal is transmitted to the ground IoT devices using beamforming technology.

[0080] The drone swarm is also connected to a low-orbit satellite, and all drones send the collected information to the satellite, which is then aggregated and processed. Each ground IoT device is also equipped with dual antennas, one for receiving the sensing signals emitted by the drone and one for transmitting the collected data to the drone. The drone information collection system has a total of M drones, for example, 10 drones, and I ground IoT devices, for example, 30 ground IoT devices, of which, They represent a collection of drones and a collection of IoT devices respectively. Multiple drones collect data from multiple ground devices. The drones perceive the locations of connected IoT devices through the angle and distance information of the echo signals.

[0081] The drone sends a dedicated sensing signal to the IoT device, is the perception signal sent by drone m to the IoT device in time slot n, In the formula, the perception signal emitted by UAV m is And the covariance matrix of the perceived signal is Represents the transmit beam precoding vector of the perception signal. Definition is the data signal sent by the i-th ground IoT device to the m-th drone in time slot n. Indicates the current position of the drone. Assuming that the drone is flying at a fixed height G, then is the two-dimensional plane coordinate of UAV m, expressed as use To represent the location information of the i-th ground IoT device, That is, the two-dimensional plane coordinates of the ground IoT device i, Then determine the channel gain β from the drone to the user m,i for: Where β0 represents the channel gain at 1 meter.

[0082] The response vector of drone m to the user's transmitted signal is: Where b is the half-wavelength margin of the antenna, λ is the carrier wavelength, θ m,i is the elevation angle of the geographic path between the mth UAV and the i-th ground IoT device, Therefore, the baseband equivalent channel from the mth drone to the i-th IoT device is: Where, d m,i represents the distance between the mth UAV and the i-th ground IoT device, where j represents a complex number.

[0083] Step S2, based on the baseband equivalent channel, determine the data signal received by the drone through the receiving antenna

[0084] In order to realize the location perception of ground IoT devices, the drone transmits the perception signal to the ground IoT device, and the ground IoT device sends the echo signal to the drone. The drone obtains the perception of the arrival angle and distance, that is, the perception information Φ = [θ m,i ,d m,i ] T In addition, the ground IoT device sends the collected data information to the drone through the transmitting antenna. Then the drone receives the information including the echo signal and the collected data signal, that is, the data signal for:

[0085] In the formula, α m,i is the radar cross-sectional area of ​​the UAV m, which satisfies the normal distribution. is the perception signal transmitted by UAV m in time slot n, represents the perception signal emitted by UAV m, and represents the transmit beam precoding vector of the perception signal, and The data sent by the ground IoT device i to the drone m. represents the receive beam coding vector of the data signal, and The data signal sent by the ground IoT device to the drone is modeled as an independent Gaussian random signal with zero mean and unit power, satisfying z m,i[n] represents additive Gaussian white noise, satisfying The superscript T indicates transpose.

[0086] Furthermore, since the sensing information is obtained by the echo of the transmitted signal from the UAV, the parameters that need to be sensed are Where Φ=[θ m,i ,d m,i ] T , In order to obtain the CRB (Cramér-Rao Bound, abbreviated as CRB) index of the perception parameter (in statistical models, it is often used to evaluate the accuracy of the estimation quantity), the echo signal received by the drone is vectorized. The data signal transmitted by the ground IoT device to the drone is also regarded as noise, that is, Then the data signal received by the drone is:

[0087] Then make Then Convert to: Y = αHX + V. Continue to convert the expression into vector form, let η=vec{αHX}, The signal received by the drone is a Gaussian observation that obeys the normal distribution in the complex domain, that is, in Then its Fisher information matrix F IM The element in the mth row and ith column is represented as: Among them, ξ m and i represents the m-th and i-th elements of ξ.

[0088] The Fisher information matrix is ​​further divided into a 2x2 matrix, that is, In the formula, F ΦΦT represents the matrix for finding the second partial derivative of the parameter Φ, Indicates the parameters Find the matrix of the second partial derivative, Represents the parameters Φ and Find the matrix of partial derivatives respectively.

[0089] Then, according to the relationship between the Fisher information matrix and the CRB matrix, the CRB estimated for parameter Φ is:

[0090] Step S3, determine the data rate R of the mth UAV collecting the i-th ground IoT device m,i [n].

[0091] The signal-to-interference-to-noise ratio of the drone data signal is: Then the rate at which the mth drone collects the ith ground IoT device can be expressed as: Rm,i [n] = log2(1 + γ m,i [n]).

[0092] Step S4, establishing an optimization problem based on the constraints and the data rate.

[0093] Under the conditions of perception accuracy and power constraints, the UAV information collection system can maximize the benefits of the entire system by optimizing the UAV trajectory, beamforming matrix, and perception signals, thereby realizing information collection services in a high transmission rate and high efficiency. Specifically, the benefit of the UAV information collection system is defined as the sum of the reachable rates of all links in the system. The goal is to optimize the maximum rate at which UAVs collect information from ground IoT devices while ensuring perception accuracy.

[0094] The optimization problem is then expressed as:

[0095] The constraints are:

[0096] In the formula, the constraint C1 represents the trajectory constraint of all drones, C2 represents the power constraint of drones, C3 represents the perception accuracy requirement of drones, C4 represents the minimum distance constraint between drones to avoid collision, S represents the radius of the entire coverage area, P represents the power upper limit, Γ represents the perception accuracy limit, and L min It represents the minimum separation distance between drones, and k represents all drones except drone m.

[0097] Step S5, solving the optimization problem through a multi-agent reinforcement learning model to determine the maximum total rate at which M drones collect information from I ground IoT device.

[0098] After the optimization problem is determined, a multi-agent reinforcement learning method can be used to solve the optimization problem. During training and learning, all drones send the collected information to the satellite, where centralized learning is performed. The state space, action space, and rewards that adapt to the environment are designed to represent different states, feasible actions, and evaluation indicators. Then, a multi-agent reinforcement learning algorithm is used to explore the state space, select the best action, and obtain the maximum reward. Finally, a solution that maximizes the benefits of the entire drone information collection system is obtained.

[0099] The present invention considers the ISAC system scenario including multiple drones and multiple ground IoT devices, and establishes an optimization problem for the constructed drone information collection system. While meeting the perception accuracy requirements, drone trajectory restrictions, power restrictions, and distance restrictions between drones, the drone trajectory, transmission beamforming, and perception signals are jointly optimized to achieve high-speed signal transmission and efficient information collection, so as to maximize the total transmission rate of the drone information collection system. In order to better reflect the mutual assistance of multiple agents in actual scenarios and avoid interference between agents, the present invention also combines a multi-agent deep reinforcement learning method to obtain the optimal agent action setting for the transmission rate of the drone information collection system, thereby realizing the overall path optimization of the drone group.

[0100] In some optional implementations, the CRB(Φ) estimated by the parameter Φ is determined by the following steps:

[0101] Determine the parameters that need to be sensed Where Φ=[θ m,i ,d m,i ] T ,

[0102] The echo signal received by the drone is vectorized, and the data signal transmitted by the ground IoT device to the drone is regarded as noise, that is,

[0103] Determine the data signal received by the drone:

[0104] Will Convert to: Y = αHX + V, where

[0105] Vectorize Y = αHX + V to determine η=vec{αHX}, in, The Fisher information matrix F IM The element in the mth row and ith column is represented as: Among them, ξ m and i represents the mth and ith elements of ξ;

[0106] Divide the Fisher information matrix into a 2x2 matrix F IM ,in, In the formula, F ΦΦT represents the matrix for finding the second partial derivative of the parameter Φ, Indicates the parameters Find the matrix of the second partial derivative, Represents the parameters Φ and Find the matrix of partial derivatives respectively;

[0107] According to the relationship between the Fisher information matrix and the CRB matrix, the CRB estimated for the parameter Φ is:

[0108] Please refer to the specific description of step S2 above for details, which will not be repeated here.

[0109] In this embodiment, by determining the CRB index, it is possible to assist in designing a better estimation algorithm and optimize the signal processing accuracy.

[0110] In some optional implementations, solving the optimization problem through a multi-agent reinforcement learning model to determine the maximum total rate at which M drones collect information from I ground IoT device includes:

[0111] Determine the M drones as M intelligent agents, and determine the state set, observation set, and action set corresponding to each intelligent agent.

[0112] Determine the strategy corresponding to each agent based on the state set, observation set, and action set.

[0113] Based on the optimization problem, determine the reward function, where the reward function is:

[0114]

[0115] In the formula, β represents the penalty factor, p1, p2, p3, and p4 are all penalty items, where:

[0116]

[0117] In the formula, F(f(X)) represents the discriminant value after being judged by the discriminant function f(X), where:

[0118]

[0119] In the formula, when K=1, it represents the reward function value corresponding to the penalty term p1, and so on.

[0120] Get a pre-built policy decision model, which includes an Actor-Critic network.

[0121] Based on the reward function and the strategy decision model, a decision is made for each strategy to determine the optimal strategy so that the total rate at which M drones collect information from I ground IoT device is maximized.

[0122] In some optional embodiments, the Actor network in the Actor-Critic network is used to make an action decision for each observation according to the policy, and the Critic network is used to evaluate the action selected by the Actor network using a state-value function;

[0123] Among them, the state-value function is:

[0124]

[0125] In the formula, γ represents the discount factor, τ represents the power parameter of the discount factor, and r m represents the reward of agent m, represents the input state of agent m, a m Represents the action corresponding to the input state. In the formula, the reward r m Corresponding to the reward function

[0126] Specifically, in this embodiment, the problem is first converted into a distributed observable Markov game, and then a multi-agent deep deterministic policy gradient algorithm (Multi-Agent Deep Deterministic Policy Gradient, MADDPG for short) is used to solve the problem.

[0127] The problem is converted into a distributed partially observable Markov game of M agents, where the M agents are M drones. Now define the Markov game of M agents as a set of states A set of observations is A set of actions state Includes drone position. Action is the action space of all agents. In each time slot n, each agent observes is part of the state. For a given state The mth agent uses strategy π according to its observations m :b m →a m To select the action space, that is, to determine the strategy corresponding to each agent based on the state set, observation set and action set.

[0128] At time slot n, the state of the environment is expressed as:

[0129] According to the current strategy π m :b m →a m , each drone selects an action from the action space. The drone action includes the flight direction of the drone The flight distance of the drone d m [n], beamforming matrix w m [n], sensory signal s m [n]. Then in the nth time slot, the action of the mth agent can be expressed as:

[0130] The reward function is a function of state and action, which is used to measure the effectiveness of an action taken by the agent in a given state. In the training phase, within a time slot n, the agent selects an action, and the corresponding reward will be returned to the agent. The agent updates its strategy based on the received reward to find an optimal strategy. The optimal strategy is to select the action with the highest reward under different environmental states.

[0131] Based on the optimization problem, the reward function is designed as:

[0132] If the agent violates any constraint in the optimization problem, the corresponding penalty term will be added to the reward. p1, p2, p3, and p4 represent the penalty terms for the three constraints in the optimization problem. β represents the penalty factor. Definition

[0133] F(f(X)) represents the discriminant value after being judged by the discriminant function f(X), where:

[0134]

[0135] The penalty terms p1, p2, p3, p4 are expressed as:

[0136]

[0137] Then obtain the policy decision model. The training network contains Actor and Critic networks. The policy-based Actor network is based on the policy π m For each observation, an action decision is made. The value-based Critic network uses the state-value function H(·) to evaluate the action selected by the Actor network. The input state is The discount factor is γ, the discount factor power parameter is τ,

[0138] The corresponding reward is r m , then the state-value function is expressed as:

[0139]

[0140] In order to better reflect the impact of constraints on system benefits and the mutual influence between drones in the scene, in this embodiment, a multi-agent deep deterministic policy gradient algorithm (Multi-Agent Deep Deterministic Policy Gradient, MADDPG) with centralized training and distributed execution is used. Consider using centralized training and distributed execution of MADDPG to solve optimization problems in multi-drone scenarios. Drones cannot communicate with each other. In centralized training, low-Earth satellites obtain strategies by learning the experience training of all drone agents. In distributed execution, after offline learning, satellites forward the learned strategies and state-value functions to the corresponding drones for distributed execution. In centralized training, the observations and actions of all agents are known; in distributed execution, each agent only has local observations, and obtains its own actions in the current state through its own Actor network in the absence of other agents. And in the invention, all agents share a reward function. That is, while meeting the requirements of perception accuracy, UAV trajectory restrictions, power restrictions and the distance between UAVs, the UAV trajectory, transmit beamforming and perception signals are jointly optimized to achieve high-speed signal transmission and efficient information collection, maximizing the total transmission rate of the UAV information collection system.

[0141] In some optional implementations, the Actor-Critic network determines the optimal strategy by adjusting the parameters of the Actor network and the Critic network; wherein the Actor-Critic network is composed of an evaluation network and a target network, and the evaluation network includes the parameters of the Actor network And the parameters of the Critic network The target network includes the parameters of the Actor network And the parameters of the Critic network

[0142] The evaluation network parameters of the critic network are adjusted by reducing the loss, and the lth state transfer The loss function is: Among them, H′(·) is the state value function of the target network. Under continuous and differentiable conditions, the Critic network is adjusted according to the gradient of the loss function;

[0143] The evaluation network parameters of the Actor network are updated by maximizing the policy objective function, which is: In the formula, χ m (·) is the evaluation network function of the Actor network;

[0144] The target network parameters of the Critic network are updated using the first function, which is:

[0145] The target network parameters of the Actor network are updated using the second function, which is: In the formula, κ is a positive constant close to 0.

[0146] In some optional implementations, the Actor-Critic network adjusts the parameters of the Actor network and the Critic network, including:

[0147] Initialize the evaluation network parameters and target network parameters of the Actor network and Critic network corresponding to each agent;

[0148] Initialize the experience pool size of each agent;

[0149] Determine time slot n and agent m;

[0150] Observe the environment of agent m and obtain state s m [n];

[0151] According to the status m [n], agent m chooses to perform action a m [n];

[0152] After agent m performs an action, it obtains the next time slot state s m [n+1], until all actions a of agent m m [n] The execution is completed, and all states, actions, and rewards are stored in the experience pool;

[0153] Select experience samples from the experience pool;

[0154] Based on the empirical samples, the evaluation network parameters of the Critic network are updated by minimizing the loss function, the evaluation network parameters of the Actor network are updated by maximizing the strategy objective function, the target network parameters of the Critic network are updated by the first function, and the target network parameters of the Actor network are updated by the second function.

[0155] The main idea of ​​MADDPG is to adjust the parameters of the Actor network and the Critic network to obtain the optimal strategy. The Actor network and the Critic network are composed of an evaluation network (the parameters of the Actor network And the parameters of the Critic network ) and a target network (the parameters of the Actor network And the parameters of the Critic network )constitute.

[0156] Using an offline learning strategy, an experience replay container of size D is used to store transfer information, including the current state, action, reward, and state at the next moment. The parameters of the evaluation network are updated in real time. The update process is to first randomly extract a small batch of sample experiences from the experience replay container and input them into the agent one by one. During the training phase, the Actor and Critic networks update the parameters of the evaluation network based on each input small batch of samples. In the lth transfer, the Critic network adjusts the parameters of the evaluation network by minimizing the loss. The loss function is expressed as:

[0157] Then if Continuous and differentiable, the Critic network can be adjusted according to the gradient of the loss function. Because the Actor network needs to make action decisions for each observation, and each agent aims to maximize the cumulative reward, the evaluation network parameters of the Actor network are updated by maximizing the policy objective function. The policy objective function is expressed as:

[0158] Among them, χ m (·) is the evaluation network function of the Actor network, which represents the deterministic policy π m :b m →a m Since the action space is continuous and D is continuously differentiable, we can update Adjustment As the evaluation network parameters are continuously updated, the target network adopts soft update:

[0159]

[0160] Here, κ is a positive constant close to 0.

[0161] In this embodiment, the specific multi-agent deep deterministic policy gradient algorithm is as follows:

[0162]

[0163] The present invention fully considers the ISAC system to optimize the achievable rate of the transmission link between the drone and the ground Internet of Things under power constraints, drone trajectory design and perception accuracy constraints. Due to the complex environment, multi-agent reinforcement learning is used to fully explore the environment and consider the decision-making interaction between drones. Unlike traditional technologies, each drone supports ISAC technology and can perceive the location information of ground Internet of Things devices. Specifically, the limitation of perception accuracy will affect the transmission performance of the system, that is, it is necessary to balance the perception accuracy and transmission rate, and seek the optimal solution of the communication rate that meets the perception accuracy requirements through autonomous learning of the agent. The centralized learning distributed execution method is adopted, which fully considers the interference of the mutual influence between agents on the performance of the entire system, and greatly improves the convergence performance of the drone information collection system.

[0164] The present invention establishes an optimization problem for the performance requirements of the UAV information collection system. This problem optimizes the trajectory, transmission beamforming and sending symbols of the UAV while meeting the UAV trajectory restriction, power restriction and perception accuracy requirements, thereby maximizing the transmission rate of the UAV information collection system.

[0165] In this embodiment, a rate optimization device for a drone information collection system based on synaesthesia integration is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0166] This embodiment provides a rate optimization device for a drone information collection system based on synaesthesia integration, such as Figure 3 As shown, the device comprises:

[0167] The baseband equivalent channel determination module 101 is used to determine the baseband equivalent channel from the drone to the ground IoT device; including: determining the channel gain β from the drone to the user m,i , In the formula, represents the two-dimensional plane position of the UAV m at height G, Represents the two-dimensional coordinate information of the i-th IoT device, β0 represents the channel gain at 1 meter, n is the time slot, and the total number of time slots is N; it is also used to determine the baseband equivalent channel h from the mth drone to the i-th ground IoT device based on the channel gain. m,i ,in, Where, d m,i represents the distance between the mth UAV and the ith ground IoT device. The superscript H represents the conjugate transpose. is the response vector of the UAV m to the user’s transmitted signal, where b is the antenna half-wavelength margin, λ is the carrier wavelength, and N t Represents the total number of transmitting antennas of the drone. The number of transmitting antennas is the same as that of receiving antennas. m,i is the elevation angle of the geographic path between the mth UAV and the i-th ground IoT device,

[0168] The data signal determination module 102 is used to determine the data signal received by the drone through the receiving antenna based on the baseband equivalent channel. In the formula, α m,i is the radar cross-sectional area of ​​the UAV m, which satisfies the normal distribution. is the perception signal transmitted by UAV m in time slot n, represents the perception signal emitted by UAV m, and represents the transmit beam precoding vector of the perception signal, and The data sent by the ground IoT device i to the drone m. represents the receive beam coding vector of the data signal, and z m,i [n] represents additive Gaussian white noise, satisfying The superscript T indicates transposition;

[0169] The rate determination module 103 is used to determine the data rate R of the mth UAV collecting the ith ground IoT device. m,i [n], R m,i [n] = log2(1 + γ m,i [n]), where

[0170] The establishment module 104 is used to establish an optimization problem based on the constraint conditions and the data rate; wherein the optimization problem is: In the formula, is the covariance matrix of the perception signal, M is the total number of drones, and I is the total number of ground IoT devices;

[0171] The constraints include:

[0172] In the formula, C1 represents the trajectory constraint for all drones, C2 represents the power constraint for drones and ground IoT devices, C3 represents the perception accuracy requirement for drones, C4 represents the minimum distance constraint between drones to avoid collision, S represents the area of ​​the entire scene, P represents the power upper limit, Γ represents the perception accuracy limit, and L min It represents the minimum separation distance between drones, and k represents drones other than drone m;

[0173] The optimization module 105 is used to solve the optimization problem through a multi-agent reinforcement learning model to determine the maximum total rate at which M drones collect information from I ground IoT device.

[0174] The rate optimization device of the drone information collection system based on synaesthesia integration in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0175] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0176] The embodiment of the present invention also provides a computer device having the above Figure 3 The rate optimization device of the UAV information collection system based on synaesthesia integration is shown.

[0177] See also Figure 4 , Figure 4 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 4 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 4 A processor 10 is taken as an example.

[0178] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0179] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0180] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0181] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0182] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0183] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0184] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0185] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A rate optimization method for a UAV information collection system based on synaesthesia integration, characterized in that: The method comprises: Step S1, determining the baseband equivalent channel from the drone to the ground IoT device; wherein step S1 includes: Step S11, determine the channel gain β from the drone to the user m,i , In the formula, represents the two-dimensional plane position of the UAV m at height G, Represents the two-dimensional coordinate information of the i-th IoT device, β0 represents the channel gain at 1 meter, n is the time slot, and the total number of time slots is N; Step S12: Based on the channel gain, determine the baseband equivalent channel h from the mth UAV to the i-th ground IoT device. m,i ,in, Where, d m,i represents the distance between the mth UAV and the ith ground IoT device. The superscript H represents the conjugate transpose. is the response vector of the UAV m to the user’s transmitted signal, where b is the antenna half-wavelength margin, λ is the carrier wavelength, and N t Represents the total number of transmitting antennas of the drone. The number of transmitting antennas is the same as that of receiving antennas. m,i is the elevation angle of the geographic path between the mth UAV and the i-th ground IoT device, Step S2, based on the baseband equivalent channel, determining the data signal received by the drone through the receiving antenna In the formula, α m,i is the radar cross-sectional area of ​​the UAV m, which satisfies the normal distribution. is the perception signal transmitted by UAV m in time slot n, represents the perception signal emitted by UAV m, and represents the transmit beam precoding vector of the perception signal, and The data sent by the ground IoT device i to the drone m. represents the receive beam coding vector of the data signal, and z m,i [n] represents additive Gaussian white noise, satisfying The superscript T indicates transposition; Step S3, determine the data rate R of the mth UAV collecting the i-th ground IoT device m,i [n], R m,i [n] = log2(1 + γ m,i [n]), where Step S4, establishing an optimization problem based on the constraint conditions and the data rate; The optimization problem is: In the formula, is the covariance matrix of the perception signal, M is the total number of drones, and I is the total number of ground IoT devices; The constraints include: In the formula, C1 represents the trajectory constraint for all drones, C2 represents the power constraint for drones, C3 represents the perception accuracy requirement for drones, C4 represents the minimum distance constraint between drones to avoid collision, S represents the radius of the entire coverage area, P represents the power upper limit, Γ represents the perception accuracy limit, and L min It represents the minimum separation distance between drones, and k represents drones other than drone m; Step S5, solving the optimization problem through a multi-agent reinforcement learning model to determine the maximum total rate at which M drones collect information from I ground IoT device.

2. The method according to claim 1, characterized in that: Determine CRB(Φ) for the parameter Φ estimate by the following steps: Determine the parameters that need to be sensed Where Φ=[θ m,i ,d m,i ] T , The echo signal received by the drone is vectorized, and the data signal transmitted by the ground IoT device to the drone is regarded as noise, that is, Determine the data signal received by the drone: Will Convert to: Y = αHX + V, where Vectorize Y = αHX + V to determine η=vec{αHX}, in, The Fisher information matrix F IM The element in the mth row and ith column is represented as: Among them, ξ m and i represents the mth and ith elements of ξ; Divide the Fisher information matrix into a 2x2 matrix F IM ,in, In the formula, represents the matrix for finding the second partial derivative of the parameter Φ, Indicates the parameters Find the matrix of the second partial derivative, Represents the parameters Φ and Find the matrix of partial derivatives respectively; According to the relationship between the Fisher information matrix and the CRB matrix, the CRB estimated for the parameter Φ is:

3. The method according to claim 2, characterized in that The optimization problem is solved by a multi-agent reinforcement learning model to determine the maximum total rate at which M drones collect information from I ground IoT device, including: Determine the M drones as M intelligent agents, and determine the state set, observation set, and action set corresponding to each intelligent agent; Determine a strategy corresponding to each agent according to the state set, the observation set, and the action set; Based on the optimization problem, a reward function is determined, where the reward function is: In the formula, β represents the penalty factor, p1, p2, p3, and p4 are all penalty items, where: In the formula, F(f(X)) represents the discriminant value after being judged by the discriminant function f(X), where: Acquire a pre-established policy decision model, wherein the policy decision model includes an Actor-Critic network; Based on the reward function and the strategy decision model, a decision is made for each strategy to determine the optimal strategy so that the total rate at which M drones collect information from I ground IoT device is maximized.

4. The method according to claim 3, characterized in that The Actor network in the Actor-Critic network is used to make an action decision for each observation according to the strategy, and the Critic network is used to evaluate the action selected by the Actor network using a state-value function; Wherein, the state-value function is: In the formula, γ represents the discount factor, τ represents the power parameter of the discount factor, and r m represents the reward of agent m, represents the input state of agent m, a m Indicates the action corresponding to the input state.

5. The method according to claim 3, characterized in that: The Actor-Critic network determines the optimal strategy by adjusting the parameters of the Actor network and the Critic network; wherein the Actor-Critic network is composed of an evaluation network and a target network, and the evaluation network includes the parameters of the Actor network And the parameters of the Critic network The target network includes the parameters of the Actor network And the parameters of the Critic network The evaluation network parameters of the Critic network are adjusted by reducing the loss, and the lth state transfer The loss function is: Among them, H′(·) is the state value function of the target network. Under continuous and differentiable conditions, the Critic network is adjusted according to the gradient of the loss function; The evaluation network parameters of the Actor network are updated by maximizing the strategic objective function, which is: In the formula, χ m (·) is the evaluation network function of the Actor network; The target network parameters of the Critic network are updated using a first function, where the first function is: In the formula, κ is a positive constant; The target network parameters of the Actor network are updated using a second function, where the second function is:

6. The method according to claim 5, characterized in that The Actor-Critic network adjusts the parameters of the Actor network and the Critic network, including: Initialize the evaluation network parameters and the target network parameters of the Actor network and the Critic network corresponding to each agent; Initialize the experience pool size of each agent; Determine time slot n and agent m; Observe the environment of agent m and obtain state s m [n]; According to the status m [n], agent m chooses to perform action a m [n]; After agent m performs an action, it obtains the next time slot state s m [n+1], until all actions a of agent m m [n] The execution is completed, and all states, actions, and rewards are stored in the experience pool; selecting an experience sample from the experience pool; Based on the experience sample, the evaluation network parameters of the Critic network are updated by minimizing the loss function, the evaluation network parameters of the Actor network are updated by maximizing the strategy objective function, the target network parameters of the Critic network are updated by the first function, and the target network parameters of the Actor network are updated by the second function.

7. A rate optimization device for a drone information collection system based on synaesthesia integration, characterized in that: The device comprises: The baseband equivalent channel determination module is used to determine the baseband equivalent channel from the drone to the ground IoT device; it includes: a module for determining the channel gain β from the drone to the user m,i , In the formula, represents the two-dimensional plane position of the UAV m at height G, Represents the two-dimensional coordinate information of the i-th IoT device, β0 represents the channel gain at 1 meter, n is the time slot, and the total number of time slots is N; it is also used to determine the baseband equivalent channel h from the mth drone to the i-th ground IoT device based on the channel gain m,i ,in, Where, d m,i represents the distance between the mth UAV and the ith ground IoT device. The superscript H represents the conjugate transpose. is the response vector of the UAV m to the user’s transmitted signal, where b is the antenna half-wavelength margin, λ is the carrier wavelength, and N t Represents the total number of transmitting antennas of the drone. The number of transmitting antennas is the same as that of receiving antennas. m,i is the elevation angle of the geographic path between the mth UAV and the i-th ground IoT device, A data signal determination module is used to determine the data signal received by the drone through the receiving antenna based on the baseband equivalent channel. In the formula, α m,i is the radar cross-sectional area of ​​the UAV m, which satisfies the normal distribution. is the perception signal transmitted by UAV m in time slot n, represents the perception signal emitted by UAV m, and represents the transmit beam precoding vector of the perception signal, and The data sent by the ground IoT device i to the drone m. represents the receive beam coding vector of the data signal, and z m,i [n] represents additive Gaussian white noise, satisfying The superscript T indicates transposition; The rate determination module is used to determine the data rate R of the mth UAV collecting the i-th ground IoT device m,i [n], R m,i [n] = log2(1 + γ m,i [n]), where Establish a module, for establishing an optimization problem based on the constraint conditions and the data rate; wherein the optimization problem is: In the formula, is the covariance matrix of the perception signal, M is the total number of drones, and I is the total number of ground IoT devices; The constraints include: In the formula, C1 represents the trajectory constraint of all drones, C2 represents the power constraint of drones, C3 represents the perception accuracy requirement of drones, C4 represents the minimum distance constraint between drones to avoid collision, S represents the radius of the entire coverage area, P represents the power upper limit, Γ represents the perception accuracy limit, and L min It represents the minimum separation distance between drones, and k represents drones other than drone m; The optimization module is used to solve the optimization problem through a multi-agent reinforcement learning model to determine the maximum total rate at which M drones collect information from I ground IoT device.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the rate optimization method of the unmanned aerial vehicle information collection system based on synaesthesia integration as described in any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the rate optimization method of the drone information collection system based on synaesthesia integration as described in any one of claims 1-6.

10. A computer program product, characterized in that It includes computer instructions, and the computer instructions are used to enable a computer to execute the rate optimization method of a drone information collection system based on synaesthesia integration described in any one of claims 1 to 6.

Images