Communication sensing integrated distributed unmanned aerial vehicle communication system and method

Abstract

The invention provides a communication-sensing integrated distributed unmanned aerial vehicle communication system, which realizes flexible resource allocation of sensing and communication tasks in different communication scenes by designing time-division communication-sensing integrated signals, ensures that the sensing and communication tasks do not interfere with each other, and can dynamically adjust the sensing frequency to adapt to the change of an environment state. Therefore, the communication performance and the resource utilization efficiency are improved. A plurality of unmanned aerial vehicles are trained into an intelligent agent through reinforcement learning so as to regulate and control the positions of the unmanned aerial vehicles and the communication state with a user in real time according to the state of a target environment, so that the stability and reliability of a user communication network in the target environment are ensured. Besides, a reward function oriented to cooperation of multiple unmanned aerial vehicles is designed, information exchange and cooperation among the unmanned aerial vehicles are promoted, and the total communication rate of ground users is improved to the maximum extent. In general, the invention provides a solution which is efficient in communication, flexible in deployment and stable in operation for a multi-unmanned aerial vehicle communication system.

Description

Technical Field

[0001] The present invention relates to the fields of wireless communication technology and computer technology, and more specifically to the field of multi-UAV assisted ground user communication, and more specifically to a synaesthesia integrated distributed UAV communication system and method. Background Art

[0002] Synaesthesia integration has become a key technology for future wireless networks. Unlike separate radar perception systems and communication systems, communication and perception in synaesthesia integration share the same hardware infrastructure. While providing communication services to communication users, it extracts information such as user location and movement speed from echo signals, and further uses perception information to assist communication resource scheduling. However, the perception performance of synaesthesia integration is closely related to the channel environment between the transmitter and the receiver. Long distances and obstacles will cause serious echo signal loss, which seriously degrades the perception performance. UAVs have flexible 3D mobility and high-altitude flight characteristics. They can be quickly deployed in wireless communication scenarios such as disaster relief, emergency communications, and military communications, providing ground users with good line-of-sight links and significantly improving the coverage and performance of communication networks. Therefore, UAVs are regarded as aerial synaesthesia integration platforms with great potential, which can provide more controllable and efficient communication and perception services. However, traditional work on multi-UAV communication networks focuses on the separate design of perception or communication systems, such as optimizing technical solutions with the goal of improving perception accuracy or maximizing communication rate. Different from individually designed perception or communication systems, the design of UAV synaesthesia integrated system is affected by many complex factors, including synaesthesia integrated waveform design, allocation of perception and communication bandwidth resources, UAV trajectory optimization, etc., which brings new challenges to multi-UAV scheduling under the synaesthesia integrated system.

[0003] The next generation 6G mobile communication network will go beyond traditional communication services and provide users with additional high-precision perception services such as user positioning and navigation. However, in traditional wireless communication systems, communication and perception functions are often separated, independent in hardware composition and software design, and have low resource utilization. Synaesthesia integration technology is considered to be a promising solution to reduce hardware costs and improve spectrum efficiency by sharing common hardware and signals to simultaneously transmit communication data and perceive on the same spectrum. Synaesthesia integration platforms built on ground base stations are susceptible to interference from obstacles, which affects perception performance. Unlike traditional ground base station cellular networks fixed on the ground, synaesthesia integration platforms based on drones have additional design freedom. By optimizing the position of drones and their flight trajectories, the channel state of communication users and perception targets is enhanced, and the performance of synaesthesia integration services is improved. Existing multi-drone deployment optimization schemes based on synaesthesia integration can be roughly divided into two categories, namely static drone deployment and mobile drone trajectory design. When wireless communication is carried out through static drone deployment, optimization methods such as topology and clustering are used to maximize the coverage of drones by optimizing the position and power allocation of multiple drones. The mobile drone improves the performance of synaesthesia integration by optimizing the flight trajectory, bringing the drone closer to the mobile user and the target of interest.

[0004] However, whether it is the deployment of static drones or the trajectory optimization of mobile drones, when performing perception and communication tasks, it is often assumed that perception and communication are performed in a fixed cycle, which may ignore the asymmetric perception and communication requirements in the actual system. For example, in a scenario where the environmental state changes rapidly (vehicles driving at high speed), the system needs to perform perception tasks at a higher frequency to obtain the location, speed and other information of the communication user in a timely manner. For scenarios where the environmental state changes slowly (concerts with dense crowds), the location of the communication user remains basically unchanged, and the system only needs to provide a stable communication connection without the need for high-frequency perception of the user. The existing drone-based synaesthesia integration technology solution does not take into account the impact of the perception frequency on the system.

[0005] In summary, when using distributed multi-UAV systems to provide communication services, existing technologies face the challenge of sharing limited resources between communication and perception tasks in the same period of time. Since communication and perception share the same time-frequency resources, frequent perception activities may weaken communication performance. In addition, the location configuration of UAVs, the establishment of user associations, and bandwidth allocation all directly affect the communication performance of the system, and the requirements of these factors in different scenarios are also different. Traditional optimization methods are difficult to solve this non-convex optimization problem.

[0006] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. If there is no evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art. Summary of the invention

[0007] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a synaesthesia integrated distributed UAV communication system and method.

[0008] The objective of the present invention is achieved through the following technical solutions:

[0009] According to a first aspect of the present invention, a synaesthesia-integrated distributed UAV communication system is proposed, which is used to provide communication services for users in a target environment. The system includes a plurality of distributed UAVs, wherein all the UAVs are configured to share spectrum, hardware, and signal resources, and provide communication services for users in the target environment in each perception cycle in the following manner: each UAV sends a synaesthesia-integrated signal to users within its coverage area for communication to obtain target environment information within its coverage area, wherein the synaesthesia-integrated signal is inserted into a corresponding communication frame according to the perception cycle; a system state is constructed based on the target environment information obtained by all UAVs and the position information of the UAVs, and a pre-trained decision model is used to obtain a scheduling strategy for the UAV based on the system state, wherein the scheduling strategy indicates the position adjustment that the UAV needs to make and the communication status with the user; the decision model takes the state as input and the scheduling strategy as output and is obtained based on reinforcement learning training; all UAVs adjust their positions and the communication status with ground users according to the scheduling strategy output by the decision model.

[0010] According to the first aspect of the present invention, the synaesthesia integrated signal allocates time division resources based on time division multiple access communication technology and changes the frequency of acquiring target environment information by adjusting the time frame ratio of perception signal and communication signal in the synaesthesia integrated signal within the same time period.

[0011] According to the first aspect of the present invention, the decision model is an intelligent agent obtained based on reinforcement learning, with the drone's location information and target environment information as states, and drone position adjustment, perception cycle adjustment and ground user's communication status adjustment as actions.

[0012] According to the first aspect of the present invention, the decision model is trained based on reinforcement learning through the following steps: step S1, constructing an action reward table for each drone, wherein each action reward table is used to save the reward values ​​that the corresponding drone can obtain when performing different actions in different states; step S2, repeating the following steps according to a preset number of times; step S21, each drone randomly explores an action based on the current state of the system as the first alternative action; each drone obtains the action with the largest reward value from the action reward table as the second alternative action based on the current state of the system; step S21, obtaining the probability of the first alternative action being selected and the probability of the second alternative action being selected in the current round, and each drone selects an action from its two alternative actions as a subsequent action based on this probability; step S22, each drone executes the corresponding subsequent action and updates the state, and obtains the reward corresponding to each subsequent action and updates the corresponding action reward table accordingly.

[0013] According to the first aspect of the present invention, the subsequent action is a combination of the following four operations: adjusting the perception period, which is to adjust the ratio of the time frame occupied by the communication signal and the perception signal in the synaesthesia integrated signal within the same time period to change the perception period for obtaining the target environment information; adjusting the position of the drone, which is to adjust the position of the drone in the target environment; adjusting the service status, which is to adjust the communication service status provided by the drone to each user; adjusting the bandwidth, which is to adjust the communication bandwidth allocated to each drone for each user.

[0014] According to the first aspect of the present invention, the probability of the first candidate action being selected and the probability of the second candidate action being selected are obtained in the following manner:

[0015]

[0016] in, represents the iteration round, Indicates The probability of the first alternative action being selected in the round, represents the initial probability of selecting the first alternative action, represents the minimum probability of selecting the first alternative action. is the attenuation parameter, The probability of the second alternative action being selected is .

[0017] According to a first aspect of the present invention, the action reward table of each drone is updated in the following manner: a first state and a second state are obtained, wherein the first state is the state of the system before the drone performs a subsequent action, and the second state is the state of the system after the drone performs the subsequent action; an instant reward value obtained by the drone performing the subsequent action is determined based on a preset reward function; and an expected reward that can be obtained by the drone performing the subsequent action is determined in the following manner:

[0018]

[0019] in, It represents the expected reward that the drone can obtain after performing subsequent actions. Indicates The subsequent actions of the drone, Indicates The instant reward value obtained by the drone for executing subsequent actions, Indicates the second state, Indicates In the action reward table of the drone, the drone can obtain the action corresponding to the maximum reward value in the second state. Indicates The maximum reward value that the drone can obtain in the second state in the action reward table of the drone. Represents the discount factor; based on the immediate reward and expected reward, the reward value obtained by the drone for subsequent actions is obtained in the following way:

[0020]

[0021] in, represents the reward value obtained by the drone performing subsequent actions in the first state, It indicates the reward value corresponding to the subsequent action performed by the drone in the first state saved in the reward table. Represents a preset learning rate; based on the obtained reward value obtained by the drone performing subsequent actions in the first state, the reward value stored in the reward table for the drone performing subsequent actions in the first state is replaced to update the reward table of the drone.

[0022] According to the first aspect of the present invention, the preset reward function is configured as:

[0023]

[0024]

[0025] in, Indicates Instant reward for drones, represents the total number of drones, Indicates The first drone and The Euclidean distance between the two drones, Indicates The first drone and The reward priority calculation function between drones, It is The total communication data rate provided by the UAV to users in the target environment, Indicates the total number of users in the target environment, Indicates The first drone The relationship between users The drone flew to When providing communication services to individual users, On the contrary, when The drone did not When providing communication services to individual users, , Indicates The first drone The communication bandwidth allocated to each user, Indicates The first drone The signal-to-interference-noise ratio between users.

[0026] According to the first aspect of the present invention, the reward priority calculation function is configured as follows:

[0027]

[0028] in, , Indicates the serial number of the drone, Indicates the communication radius of the drone’s ground coverage. Indicates The first drone Euclidean distance between drones.

[0029] According to a second aspect of the present invention, a synesthesia-integrated distributed UAV communication method is proposed for providing communication services to users in a target environment, the method comprising: obtaining a synesthesia-integrated distributed UAV communication system as described in any one of the first aspects of the present invention; deploying the obtained synesthesia-integrated distributed UAV communication system to the target environment to provide communication services to all users in the target environment.

[0030] Compared with the prior art, the advantages of the present invention are:

[0031] The present invention proposes a synaesthesia integrated distributed UAV communication system, which realizes flexible resource allocation for perception and communication tasks in different communication scenarios by designing time-division synaesthesia integrated signals, ensures that perception tasks and communication tasks do not interfere with each other, and can dynamically adjust the perception frequency to adapt to changes in environmental conditions, thereby improving communication performance and resource utilization efficiency. Through reinforcement learning, multiple UAVs are trained as multiple intelligent agents to regulate the position of the UAV and the communication status with the user in real time according to the state of the target environment, thereby ensuring the stability and reliability of the user communication network in the target environment. In addition, by designing a reward function for multi-UAV collaboration, information exchange and collaborative cooperation between UAVs are promoted, the UAV coverage of the ground is optimized, and the total communication rate of ground users is maximized. In general, the present invention provides a solution for multi-UAV communication systems with efficient communication, flexible deployment and stable operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0033] Figure 1 This is an architecture diagram of a synaesthesia-integrated distributed UAV communication system according to an embodiment of the present invention;

[0034] Figure 2 A time-division synaesthesia integrated frame structure according to an embodiment of the present invention;

[0035] Figure 3 is a relationship diagram between the number of iterations and the total communication data rate according to an embodiment of the present invention;

[0036] Figure 4 4 is a comparison chart of total communication data rates according to a comparative experiment of an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail by specific embodiments below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0038] As mentioned in the background technology section, the existing technology can obtain the user's position, speed and other status information when providing communication services in a multi-UAV system based on time-division synaesthesia integrated signals. However, in the time-division synaesthesia integrated signal, communication and perception tasks compete for limited resources in the same period, and communication and perception tasks share the same time-frequency resources. Frequent perception will reduce communication performance. This causes the performance between the two to affect each other. The asymmetry of perception and communication requirements in actual scenarios, different scenarios have different requirements for perception cycles, and the location deployment of drones, the establishment of association relationships with users, and the bandwidth allocation scheme are tightly coupled and directly related to the system communication performance. Traditional distributed drone communication systems are difficult to solve this non-convex problem.

[0039] To solve the above problems, the present invention proposes a synaesthesia integrated distributed UAV communication system. The system realizes flexible allocation of resources for perception and communication tasks in different communication scenarios by designing time-division synaesthesia integrated signals, ensures that perception tasks and communication tasks do not interfere with each other, and can dynamically adjust the perception frequency to adapt to changes in environmental conditions, thereby improving communication performance and resource utilization efficiency. Through reinforcement learning, multiple UAVs are trained as multiple intelligent agents to regulate the position of the UAV and the communication status with the user in real time according to the state of the target environment, thereby ensuring the stability and reliability of the user communication network in the target environment. In addition, by designing a reward function for multi-UAV collaboration, information exchange and collaborative cooperation between UAVs are promoted, the coverage of the UAV to the ground is optimized, and the total communication rate of ground users is maximized. In general, the present invention provides a solution for multi-UAV communication systems with efficient communication, flexible deployment and stable operation.

[0040] In summary, the synaesthesia integrated distributed UAV communication system is used to provide communication services for users in the target environment. The architecture of the system is as follows: Figure 1 As shown, the system includes a plurality of distributed UAVs, wherein all the UAVs are configured to share spectrum, hardware, and signal resources, and provide communication services to users in the target environment in each perception cycle in the following manner: each UAV sends a synaesthesia integrated signal to users within its coverage area for communication to obtain target environment information within its coverage area, wherein the synaesthesia integrated signal is inserted into the corresponding communication frame according to the perception cycle; the state of the system is constructed based on the target environment information obtained by all UAVs and the position information of the UAVs, and a pre-trained decision model is used to obtain the scheduling strategy of the UAV based on the state, the scheduling strategy indicates the position adjustment that the UAV needs to make and the communication status with the user, and the decision model takes the state of the system as input and the scheduling strategy as output and is obtained based on reinforcement learning training; all UAVs adjust their positions and the communication status with ground users according to the scheduling strategy output by the decision model.

[0041] In order to better understand the present invention, it is described in detail below with reference to specific embodiments.

[0042] According to an embodiment of the present invention, the synaesthesia integrated signal is a time-division synaesthesia integrated signal that allocates time-division resources based on time-division multiple access communication technology. On the basis of communication data transmission, it realizes the perception of the user's position, distance, speed and other information in the target environment. The time-division synaesthesia integrated signal can change the frequency of obtaining target environment information by adjusting the time frame ratio of the perception signal and the communication signal in the same time period. The time frame structure of the time-division synaesthesia integrated signal is as follows: Figure 2 As shown in the figure, it is assumed that the mission period of the drone to provide communication services to users in the target environment is T, and the entire mission period is discretized into N time slots, each time slot is a 10ms radio frame. It is assumed that the position of the drone and the user position do not change during each radio frame to facilitate the design of the frame structure. Each radio frame can be further divided into subframes with a length of 1ms. Each subframe can be functionally divided into communication subframes and perception subframes. Communication subframes are used to transmit communication signals, and perception subframes are used to transmit perception signals. Considering the asymmetric perception and communication requirements in the scene, communication often needs to be carried out in real time, while the perception of the user position only needs to be carried out periodically. Therefore, the determination of the perception period has also become a key factor affecting the communication rate. For example, if a perception subframe is inserted in each radio frame, the system will be communicating and transmitting 9 / 10 of the time in each time slot; if a perception subframe is inserted in every two radio frames, the system will be communicating and transmitting 19 / 20 of the time in each time slot, and the communication capacity will be increased by 5.6%.

[0043] According to an embodiment of the present invention, in the synaesthesia integrated distributed UAV communication system, the communication between UAVs adopts D2D communication, which is device-to-device communication, and is a technology that allows two peer user nodes to communicate directly. This communication method does not require passing through a base station or a core network, so it can significantly reduce network load and delay, and improve spectrum efficiency and user experience.

[0044] According to one embodiment of the present invention, the target environment information acquired by the drone includes: the position of each user, the bandwidth required by each user, and the signal to noise ratio of each user to each drone, wherein the bandwidth required by each user and the signal to noise ratio of each user to each drone are transmitted to the drone through the synaesthesia integration signal, and the position of the user is calculated based on the propagation time of the synaesthesia integration signal. Specifically, the drone transmits the synaesthesia integration signal to the ground, and receives the signal reflected from the ground user. The drone estimates the distance between the drone and the user according to the propagation time of the signal. It should be noted that the user's position is a three-dimensional coordinate, so at least three drones need to be estimated to determine the user's three-dimensional coordinate. The present invention uses the estimated user position information to dynamically adjust the drone position and efficiently allocate communication resources to improve the user communication quality.

[0045] According to one embodiment of the present invention, the decision model is an intelligent agent obtained based on reinforcement learning, with the drone's location information and target environment information as states, and drone position adjustment, perception cycle adjustment and communication status adjustment with ground users as actions.

[0046] According to one embodiment of the present invention, the state of the decision model includes the location information of each drone, the location information of each user, the bandwidth required by each user, and the signal-to-interference-noise ratio of each user's communication with the drone. Assuming that the number of drones is ,The number of users in the target environment is , the state can be expressed as:

[0047]

[0048] in, Indicates the first The location information of each user (i.e., the three-dimensional coordinate location), Indicates The location information (i.e., three-dimensional coordinate position) of each drone, where the flight altitude of each drone is consistent. Indicates the The bandwidth resources required by users, Indicates The first drone The signal-to-interference-noise ratio between users.

[0049] According to one embodiment of the present invention, all actions that can be performed by each drone in different states constitute the action space of the decision model, and the action space determines the actions that the drone can choose in each state. These actions are the way the drone interacts with the environment, and determine how the drone transfers from one state to another. In the scenario of the present invention, the action performed by each drone is a combination of the following four operations: adjusting the perception cycle, which is to adjust the time frame ratio of the communication signal and the perception signal in the synaesthesia integrated signal in the same period to change the perception cycle for obtaining the target environment information; adjusting the drone position, which is to adjust the position of the drone in the target environment. Adjust the service state, which is to adjust the communication service state provided by the drone to each user; adjust the bandwidth, which is to adjust the communication bandwidth allocated to each drone for each user. The present invention takes into account the hardware and software limitations of a single drone, and the number of users that each drone can serve simultaneously is limited. At the same time, each user only establishes a communication connection with one drone when communicating.

[0050] According to one embodiment of the present invention, the decision model is trained based on reinforcement learning through the following steps: step S1, constructing an action reward table for each drone, wherein each action reward table is used to save the reward values ​​corresponding to the corresponding drone performing different actions in different states; step S2, repeating the following steps according to a preset number of times; step S21, each drone randomly explores an action based on the current state of the system as the first alternative action; each drone obtains the action with the largest reward value from the action reward table based on the current state of the system as the second alternative action; step S21, obtaining the probability of the first alternative action being selected and the probability of the second alternative action being selected in the current round, and each drone selects an action from its two alternative actions as a subsequent action based on this probability; step S22, each drone executes the corresponding subsequent action and updates the state, and obtains the reward corresponding to each drone executing the subsequent action and updates the corresponding action reward table accordingly.

[0051] According to one embodiment of the present invention, in step S1, in order to maximize the total communication data rate of all target environment users, each drone will establish an action reward table for storing reward values ​​corresponding to different actions performed by the drone in different states. For example, the reward value corresponding to the drone taking action a in state s is Q, and the larger the value of Q, the higher the benefit of taking action a in the current environment state s.

[0052] According to one embodiment of the present invention, in step S21, in order to prevent the scheduling strategy output by the decision model from falling into the local optimum, the present invention uses the ϵ-greedy strategy to select actions. This strategy is used to balance the trade-off between exploration and utilization, which means that when each drone selects an action, it must not only use the current knowledge to select the optimal action, but also have a certain probability of exploring other possible actions to avoid falling into the local optimal solution. For example, setting the probability of the first candidate action being selected ϵ=0.1 means that there is a 10% probability of randomly exploring an action based on the current state, and a 90% probability of obtaining the action with the largest reward value from the action reward table based on the current state of the system. At the same time, the value of ϵ decreases with the increase in the number of training times. As the intelligent agent's understanding of the environment gradually deepens with the progress of training, the action reward table already contains many useful strategies and knowledge. By gradually reducing ϵ, the decision model makes more use of the learned strategies and selects actions that are known to bring higher returns according to the reward value table, thereby improving the overall performance.

[0053] According to one embodiment of the present invention, the probability of the first candidate action being selected and the probability of the second candidate action being selected are obtained in the following manner, that is, the probability ϵ of the first candidate action being selected in each round is configured as:

[0054]

[0055] in, represents the iteration round, Indicates The probability of the first alternative action being selected in the round, represents the initial probability of selecting the first alternative action, represents the minimum probability of selecting the first alternative action. is the attenuation parameter, The probability of the second alternative action being selected is As the training progresses, the value of ϵ gradually decreases, that is, the probability of the first candidate action being selected gradually decreases, and the decision model is more inclined to use the useful strategies and knowledge contained in the action reward table to select the optimal action for the drone.

[0056] According to one embodiment of the present invention, the action reward table of each drone is updated in the following manner: a first state and a second state are obtained, wherein the first state is the state of the system before the drone performs a subsequent action, and the second state is the state of the system after the drone performs the subsequent action; an instant reward value obtained by the drone performing the subsequent action is determined based on a preset reward function; and an expected reward that can be obtained by the drone performing the subsequent action is determined in the following manner:

[0057]

[0058] in, It represents the expected reward that the drone can obtain after performing subsequent actions. Indicates The subsequent actions of the drone, Indicates The instant reward value obtained by the drone for executing subsequent actions, Indicates the second state, Indicates In the action reward table of the drone, the drone can obtain the action corresponding to the maximum reward value in the second state. Indicates The maximum reward value that the drone can obtain in the second state in the action reward table of the drone. Represents the discount factor; based on the immediate reward and expected reward, the reward value obtained by the drone for subsequent actions is obtained in the following way:

[0059]

[0060] in, represents the reward value obtained by the drone performing subsequent actions in the first state, It indicates the reward value corresponding to the subsequent action performed by the drone in the first state saved in the reward table. Represents the preset learning rate; based on the reward value obtained by the drone performing subsequent actions in the first state, the reward value stored in the reward table is replaced to update the drone's reward table. After each round of training, the reward value table is updated to adjust the rewards that the drone can obtain for performing different actions in different states, so as to make the decision model strategy more accurate.

[0061] According to one embodiment of the present invention, the preset reward function determines the reward that the drone can obtain after performing an action based on the total communication rate between the drone and the user and the location information of the drone. For a distributed drone communication system, in order to improve the overall performance of the system, each drone needs to cooperate through information exchange. In the present invention, drones share the reward values ​​they obtain with other neighboring drones, and the definition of proximity is set by a communication distance threshold. Then, the reward of each drone is given priority based on the relative distance between the drones. The preset reward function is configured as:

[0062]

[0063]

[0064] in, Indicates Instant reward for drones, represents the total number of drones, Indicates The first drone and The Euclidean distance between the two drones, Indicates The first drone and The reward priority calculation function between drones, It is The total communication data rate provided by the UAV to users in the target environment, Indicates the total number of users in the target environment, Indicates The first drone The relationship between users The drone flew to When providing communication services to individual users, On the contrary, when The drone did not When providing communication services to individual users, , Indicates The first drone The communication bandwidth allocated to each user, Indicates The first drone The signal-to-interference-noise ratio between users.

[0065] According to one embodiment of the present invention, the reward priority calculation function obtains the priority of sharing the surrounding drone reward value based on the Euclidean distance evaluation between drones, and its configuration is configured as follows:

[0066]

[0067] in, , Indicates the serial number of the drone, Indicates the communication radius of the drone’s ground coverage. Indicates The first drone The Euclidean distance between the drones, in particular, Indicates the reward priority of the drone to itself. The reward priority is configured as 1.

[0068] In order to more clearly illustrate the beneficial effects of the present invention, the inventors compared the communication rates of the distributed UAV communication system with synaesthesia integration based on reinforcement learning, the distributed UAV communication system based on a random algorithm, and the distributed UAV communication system based on a uniform algorithm through simulation tests to demonstrate the beneficial effects of the present invention. In the distributed UAV communication system based on a random algorithm, the UAVs are randomly distributed in the user activity area. Since there is no need to perceive the user's position, the synaesthesia integrated waveform is all used for communication transmission; in the distributed UAV communication system based on a uniform algorithm, the UAVs are uniformly distributed in the user activity area. Since there is no need to perceive the user's position, the synaesthesia integrated waveform is all used for communication transmission.

[0069] The experimental environment configuration of the above simulation test is: the user movement range is an area of ​​1km*1km, a total of 100 communication users are randomly distributed in the area, and the user movement speed is assumed to be randomly initialized in the options of {0m / s, 1.5m / s, 20m / s}. When the user moves to the boundary of the area, it will move in the opposite direction. Five drones are used to provide communication services for ground users. The total bandwidth of the system is 20MHz, and the bandwidth resolution is 200kHz, that is, the bandwidth allocated to the communication users is an integer multiple of 200kHz. The simulation parameters are summarized in Table 1.

[0070] Table 1

[0071] System Parameters Parameter settings Number of users 100 Number of drones 5 Drone flight altitude 350m User activity range 1000m*1000m Grid Width 100m Total bandwidth of drone 20MHz Bandwidth resolution 200kHz System Loss 0.1dB System NLos path loss 21dB Discount Factor 0.95 Learning Rate 0.1 Maximum number of iterations 1000

[0072] The invention first verifies the convergence of the proposed technical solution. Figure 3 It can be seen that the distributed UAV communication system with integrated synaesthesia proposed in the present invention reaches convergence after 250 iterations in the process of training the decision model based on reinforcement learning, and the total communication data rate of users reaches a high value. In order to further verify the effectiveness of the proposed scheme, it is compared with the other two distributed UAV communication systems to observe the change of the total communication data rate of communication users within 10s. During this period, the users move dynamically in the activity area, and the UAVs are dynamically adjusted according to the corresponding algorithms. The simulation results are as follows: Figure 4 By calculating the average value of the total user communication data rate within 10 seconds, the results shown in Table 2 below can be obtained:

[0073] Table 2

[0074] algorithm Total user communication data rate (average value within 10s) / bps The communication system proposed by the present invention 70802140.5119 Randomized Algorithms 40923926.845 Uniform Algorithm 59142798.859

[0075] From the data in the table, it can be found that compared with the distributed UAV communication system based on the random algorithm, the total user communication data rate of the distributed UAV communication system proposed in the present invention is increased by 73.01%, and compared with the distributed UAV communication system based on the uniform algorithm, it is increased by 19.01%. Therefore, the present invention adopts a multi-agent reinforcement learning algorithm to optimize the UAV swarm scheduling scheme, thereby improving the total communication data rate of users. The UAV swarm uses time-division synaesthesia integrated signals to provide communication services for ground users, while sensing the location, speed and other information of ground users, and reasonably allocates communication resources based on the ground user status information to ensure the communication quality in the target environment.

[0076] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0077] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0078] A computer-readable storage medium may be a tangible device that holds and stores instructions used by an instruction execution device. Computer-readable storage media may include, for example, but are not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0079] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A synaesthesia integrated distributed UAV communication system, used to provide communication services for users in a target environment, characterized in that: The system includes a plurality of distributed drones, wherein all the drones are configured to share spectrum, hardware, and signal resources, and provide communication services to users in the target environment in each sensing cycle in the following manner: Each drone sends a synaesthesia integrated signal to users within its coverage area to communicate and obtain target environment information within its coverage area, wherein the synaesthesia integrated signal is inserted into the corresponding communication frame according to the perception cycle; The state of the system is constructed based on the target environment information and the position information of the drones obtained by all drones, and a pre-trained decision model is used to obtain the scheduling strategy of the drone based on the state of the system. The scheduling strategy indicates the position adjustment that the drone needs to make and the communication status with the user. The decision model takes the state as input and the scheduling strategy as output and is obtained based on reinforcement learning training; All drones adjust their positions and communication status with ground users according to the scheduling strategy output by the decision model.

2. The system according to claim 1, characterized in that The synaesthesia integrated signal allocates time division resources based on the time division multiple access communication technology and changes the frequency of acquiring the target environment information by adjusting the time frame ratio of the perception signal and the communication signal in the synaesthesia integrated signal within the same time period.

3. The system according to claim 1, characterized in that The decision model is an intelligent agent obtained based on reinforcement learning, with the position information and target environment information of the UAV as the state, and the adjustment of the UAV position, the perception cycle adjustment and the communication state adjustment of the ground user as the action.

4. The system according to claim 3, characterized in that The decision model is trained based on reinforcement learning through the following steps: Step S1: construct an action reward table for each drone, wherein each action reward table is used to store the reward value that the corresponding drone can obtain when performing different actions in different states; Step S2, repeat the following steps according to a preset number of times; Step S21: Each drone randomly explores an action based on the current state of the system as the first candidate action; each drone obtains the action with the largest reward value from the action reward table based on the current state of the system as the second candidate action; Step S21: Obtain the probability of the first candidate action being selected and the probability of the second candidate action being selected in the current round, and each drone selects one action from its two candidate actions as a subsequent action based on the probability; Step S22: Each drone performs the corresponding subsequent action and updates the status, and obtains the reward corresponding to each subsequent action and updates the corresponding action reward table accordingly.

5. The system according to any one of claim 4, characterized in that: The subsequent action is a combination of the following four operations: Adjusting the perception cycle, which is to adjust the time frame ratio of the communication signal and the perception signal in the synaesthesia integrated signal within the same period of time to change the perception cycle for obtaining target environment information; Adjusting the position of the drone, which is to adjust the position of the drone within the target environment; Adjust the service status, which is to adjust the communication service status provided by the drone to each user; Adjust bandwidth, which is to adjust the communication bandwidth allocated by each drone to each user.

6. The system according to claim 3, characterized in that The probability of the first alternative action being selected and the probability of the second alternative action being selected are obtained as follows: in, represents the iteration round, Indicates The probability of the first alternative action being selected in the round, represents the initial probability of selecting the first alternative action, represents the minimum probability of selecting the first alternative action. is the attenuation parameter, The probability of the second alternative action being selected is .

7. The system according to claim 3, characterized in that Update the action reward table for each drone as follows: Acquire a first state and a second state, wherein the first state is the state of the system before the drone performs a subsequent action, and the second state is the state of the system after the drone performs the subsequent action; Determine the instantaneous reward value obtained by the drone for executing subsequent actions based on a preset reward function; The expected reward that the drone can obtain by performing subsequent actions is determined in the following way: in, It represents the expected reward that the drone can obtain after performing subsequent actions. Indicates The subsequent actions of the drone, Indicates The instant reward value obtained by the drone for executing subsequent actions, Indicates the second state, Indicates In the action reward table of the drone, the drone can obtain the action corresponding to the maximum reward value in the second state. Indicates The maximum reward value that the drone can obtain in the second state in the action reward table of the drone. represents the discount factor; Based on the immediate reward and expected reward, the reward value obtained by the drone for subsequent actions is obtained in the following way: in, represents the reward value obtained by the drone performing subsequent actions in the first state, It indicates the reward value corresponding to the subsequent action performed by the drone in the first state saved in the reward table. Indicates the preset learning rate; The reward value obtained by the drone performing the subsequent action in the first state stored in the reward table is replaced based on the reward value obtained by the drone performing the subsequent action in the first state to update the reward table of the drone.

8. The system according to claim 7, characterized in that The preset reward function is configured as: in, Indicates Instant reward for drones, represents the total number of drones, Indicates The first drone and The Euclidean distance between the two drones, Indicates The first drone and The reward priority calculation function between drones, It is The total communication data rate provided by the UAV to users in the target environment, Indicates the total number of users in the target environment, Indicates The first drone The relationship between users The drone flew to When providing communication services to individual users, On the contrary, when The drone did not When providing communication services to individual users, , Indicates The first drone The communication bandwidth allocated to each user, Indicates The first drone The signal-to-interference-noise ratio between users.

9. The system according to claim 8, characterized in that The reward priority calculation function is configured as: in, , Indicates the serial number of the drone, Indicates the communication radius of the drone’s ground coverage. Indicates The first drone Euclidean distance between drones.

10. A synaesthesia integrated distributed UAV communication method for providing communication services to users in a target environment, characterized in that: The method comprises: Obtain the synaesthesia integrated distributed UAV communication system as described in any one of claims 1 to 9; The acquired synaesthesia-integrated distributed UAV communication system is deployed to the target environment to provide communication services for all users in the target environment.

Images