Unmanned aerial vehicle base station position deployment method, unmanned aerial vehicle base station, equipment, medium and product

Abstract

The invention provides an unmanned aerial vehicle base station position deployment method, an unmanned aerial vehicle base station, equipment, a medium and a product, and the method comprises the steps: sensing the position information of a user, and determining the parameter configuration of the unmanned aerial vehicle base station and the user; acquiring current position information and current horizontal speed; determining the configuration parameters, the current position information and the current horizontal speed as first current state information; acquiring second current state information of an adjacent unmanned aerial vehicle base station; determining the first current state information and the second current state information as observation information; inputting the observation information into a pre-trained reinforcement learning model to obtain a new flight state; and performing position adjustment according to the new flight state. According to the unmanned aerial vehicle base station position deployment method provided by the invention, under the condition that no network side provides global user position distribution information, a decision can be quickly made to update the motion state, the flight state can be quickly adjusted to approach the user in the real-time movement process of the user, and high-quality continuous communication service is provided for the user.

Description

Technical Field

[0001] The present invention relates to the field of communication network technology, and in particular to a method for deploying a drone base station location, a drone base station, equipment, a medium and a product. Background Art

[0002] In recent years, various natural disasters and sudden disasters have increased. When ground communication equipment in disaster areas is destroyed and cannot function normally, most areas have no signal. If an emergency communication network cannot be quickly established, rescue and reconstruction work in the disaster areas will face great difficulties.

[0003] Drone base stations have the advantages of fast mobility, high flexibility and easy control, and can be used to build emergency communication networks. Therefore, drone base stations are very suitable for communication equipment deployed in disaster areas. Secondly, due to the destruction of buildings and terrain in disaster areas, the increase of ground obstacles, the complex terrain, damaged roads or collapsed buildings make it difficult to deploy traditional emergency communication vehicles. In addition, the antenna height of emergency communication vehicles is also limited, so it is difficult to achieve large-area signal coverage. Compared with emergency communication vehicles, drone base stations are not affected by ground conditions in disaster areas. Drone base stations can reach places that emergency communication vehicles cannot reach, greatly improving the signal coverage of disaster areas. Therefore, for the widespread application of drone base stations, it is imperative to study how to deploy drone base stations in disaster areas.

[0004] However, there are some challenges in the existing research on drone base station deployment and parameter configuration. Most of the current methods are based on relying on the user location distribution provided by the network side, that is, solving the deployment and parameter configuration problems when the user location is known. However, it is difficult to obtain the user's real-time location in actual emergency scenarios. Therefore, how to solve the deployment and parameter configuration of drone base stations under the condition that the user location is unknown and the user moves in real time is still a problem to be solved. Summary of the invention

[0005] The present invention provides a method for deploying a drone base station position, a drone base station, a device, a medium and a product, so as to solve the problem that the prior art relies on the user location distribution provided by the network side, and the drone base station cannot decide the optimal deployment and parameter configuration without knowing the user location.

[0006] In a first aspect, the present invention provides a method for deploying a drone base station location, comprising: Sense user location information; Based on the user location information, determining the parameter configuration of the drone base station and the user; the parameter configuration includes a first transmit beamforming for user communication, a second beamforming for user perception, association information with the user, and whether to perceive information; Get the current position information and current horizontal speed; determining the configuration parameters, the current position information and the current horizontal speed as first current state information; Acquire second current state information of a neighboring drone base station; the neighboring drone base station refers to other drone base stations other than the drone base station within a preset observation range of the drone base station; determining the first current state information and the second current state information as observation information; Inputting the observation information into a pre-trained reinforcement learning model to obtain a new flight state output by the reinforcement learning model; The position is adjusted according to the new flight status.

[0007] In one embodiment, the UAV base station is subject to flight constraints during flight; the flight constraints include a first constraint, a second constraint, a third constraint, a fourth constraint, a fifth constraint, a sixth constraint, a seventh constraint, and an eighth constraint; Among them, the first constraint is used to limit the horizontal direction of the drone base station; the second constraint is used to limit the horizontal speed of the drone base station; the third constraint is used to ensure that a safe distance is maintained between the drone base stations; the fourth constraint is used to limit the distance that the drone base station moves in the horizontal direction; the fifth constraint is used to limit the distance that the drone base station moves in the vertical direction; the sixth constraint is used to ensure the stability of flight between the drone base stations; the seventh constraint is used to limit the total power of communication and perception of the drone base station; the eighth constraint is used to ensure the perception effect of the drone base station.

[0008] In one embodiment, the reinforcement learning model is obtained by pre-training in the following manner: Initialize the action network and the judgment network; Get the current state sample and current observation sample of the drone base station sample; Based on the current observation sample, output a target action through the action network so that the drone base station sample performs the target action; After the drone base station sample performs the target action, a current reward is obtained through observation by the evaluation network; After the drone base station sample performs the target action, obtaining a next state sample and a next observation sample of the drone base station sample; Generate a current trajectory set sample based on the current state information, the current observation sample, the target action, the current reward, the next state sample and the next observation sample; Calculating an advantage estimate of the current trajectory set sample; generating an experience sample based on the current trajectory set sample and the advantage estimate; Updating the next state sample to the current state sample, and updating the next observation sample to the current observation sample, iteratively executing the step of outputting the target action through the action network based on the current observation sample, until a preset number of iterations is reached, obtaining multiple experience samples, and storing the multiple experience samples in a replay buffer; A fixed number of experience samples are randomly extracted from the replay buffer, and the action network and the judgment network are updated to obtain the pre-trained reinforcement learning model.

[0009] In one embodiment, the current reward is obtained by: Obtaining the flight power consumption of the drone base station sample in the current time slot, and the throughput of the target user communicated by the drone base station sample in the current time slot; Dividing the throughput and the flight power consumption to obtain the energy efficiency of the drone base station sample in the current time slot; Calculate a penalty value when the UAV base station sample violates a flight constraint condition during flight in the current time slot; The energy efficiency is subtracted from the penalty value to obtain the current reward.

[0010] In one embodiment, the flight power consumption of the drone base station sample in the current time slot is obtained in the following manner: Obtaining the weight of the drone base station sample, and the horizontal speed and vertical speed of the drone base station sample in the current time slot; Based on the weight of the drone base station sample, the horizontal speed and the vertical speed of the drone base station sample in the current time slot, the flight power consumption of the drone base station sample in the current time slot is determined.

[0011] In one embodiment, the throughput of the target user with which the drone base station sample communicates in the current time slot is obtained in the following manner: Determine a first occurrence probability of a line-of-sight channel between the drone base station sample and the user in the current time slot and a second occurrence probability of a non-line-of-sight channel; Determine a target channel between the drone base station sample and the target user in the current time slot based on the first occurrence probability and the second occurrence probability; Obtaining a transmit beamforming vector for the drone base station sample to communicate with the target user in the current time slot; Determining a signal-to-noise ratio when the target user communicates based on the target channel and the transmit beamforming vector; Based on the signal-to-noise ratio, a throughput of the target user in the current time slot is determined.

[0012] In a second aspect, the present invention further provides a drone base station, comprising: A sensing module, used to sense user location information; A parameter configuration determination module, used to determine the parameter configuration of the drone base station and the user based on the user location information; the parameter configuration includes a first transmit beamforming for user communication, a second beamforming for user perception, association information with the user, and whether to perceive information; A first acquisition module, used to acquire current position information and current horizontal speed; a state information determination module, configured to determine the configuration parameter, the current position information and the current horizontal speed as first current state information; A second acquisition module is used to acquire second current state information of a neighboring drone base station; the neighboring drone base station refers to other drone base stations other than the drone base station within a preset observation range of the drone base station; an observation information determination module, configured to determine the first current state information and the second current state information as observation information; A flight state prediction module, used to input the observation information into a pre-trained reinforcement learning model to obtain a new flight state output by the reinforcement learning model; A flight state adjustment module is used to adjust the position according to the new flight state.

[0013] In a third aspect, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned methods for deploying the location of a drone base station are implemented.

[0014] In a fourth aspect, the present invention also provides a storage medium, comprising a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the drone base station location deployment method as described in any of the above-mentioned methods are implemented.

[0015] In a fifth aspect, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by the processor, it implements the steps of any of the above-mentioned drone base station location deployment methods.

[0016] The drone base station location deployment method, drone base station, equipment, medium and product provided by the present invention, the drone base station first senses the current user location information when there is no network side providing the location distribution information of global users, determines the parameter configuration of the drone base station and the user according to the user location information, determines its own state information by using the parameter configuration, and simultaneously obtains the state information of other drone base stations within a preset observation range, determines its own observation information by using its own state information and the state information of surrounding drone base stations, and further combines with the reinforcement learning model to quickly make a decision to update the motion state, so as to achieve that during the real-time movement of the user, the drone base station can quickly adjust the flight state to get closer to the user, and provide the user with high-quality continuous communication services. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 It is a flow chart of the method for deploying the location of a UAV base station provided by the present invention.

[0019] Figure 2 It is a structural schematic diagram of the UAV base station provided by the present invention.

[0020] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein.

[0023] Combine the following Figure 1-Figure 3 The present invention describes the drone base station location deployment method, drone base station, equipment, medium and product provided by the present invention.

[0024] It should be noted that the method for deploying the location of the drone base station provided in the embodiment of the present invention is implemented based on the drone base station. When facing emergency scenarios such as disaster-stricken areas, a drone base station cluster is used to provide effective communication services for ground users in the disaster-stricken areas. In the process of providing communication services to ground users, each drone base station can obtain its own status and the status of other drone base stations within the preset observation range, and quickly make decisions to update the motion status and parameter configuration, so as to provide greater throughput to users close to the service, while reducing the flight energy consumption of the drone base station, continuously providing communication services to ground users, and ensuring the service quality of all users.

[0025] The embodiment of the present invention takes a drone base station as an example to describe the drone base station location deployment method. All drone base stations can use the following description to implement the drone base station location deployment method.

[0026] Combination Figure 1 , Figure 1 It is a flow chart of the method for deploying the location of a UAV base station provided by the present invention.

[0027] like Figure 1 As shown, the method includes the following: Step 101: Sense user location information; Step 102: Determine parameter configuration of the drone base station and the user based on the user location information; Step 103, obtaining current position information and current horizontal speed; Step 104: determining the configuration parameters, the current position information and the current horizontal speed as first current state information; Step 105: Obtain second current state information of the neighboring drone base station; Step 106: Determine the first current state information and the second current state information as observation information; Step 107: input the observation information into the pre-trained reinforcement learning model to obtain a new flight state output by the reinforcement learning model; Step 108: Adjust the position according to the new flight status.

[0028] It should be noted that the embodiment of the present invention uses a reinforcement learning model based on multi-agent reinforcement learning (MARL) to train each drone base station in the drone base station cluster. The reinforcement learning task is modeled, and the basic elements such as agent, state, observation, action, and reward are modeled as follows.

[0029] (1) Agent: Each drone base station can be regarded as being independently controlled by an agent. When the whole drone base station moves, the agent determines the next action to be performed based on the observation information obtained. The agent moves according to the next action to be performed. After the movement, the environment will send the next state and observation, and return the reward to the agent. The agent uses the reward returned by the environment to update its knowledge and evaluate the previous action. The above steps are continuously executed in a loop until the drone base station cluster reaches the destination.

[0030] (2) State: ,in, Indicates drone base station i The position information in the tth time slot, Indicates drone base station i The horizontal velocity in the tth time slot, Indicates drone base station i The first transmit beamforming of the user communication in the tth time slot, Indicates drone base station i The second transmit beamforming perceived by the user in the t-th time slot, Indicates drone base station i Information associated with the user.

[0031] (3) Observation: Each agent’s observation includes its own state. And other items within the preset observation range d Status of the drone base station , so the drone base station i The observation is .

[0032] (4) Action: ( t ) = { , },in, Indicates drone base station i The moving speed in the tth time slot, Indicates drone base station i The yaw angle in the tth time slot, Indicates drone base station i The first target transmit beamforming for user communication in the tth time slot, Indicates drone base station i The second target transmit beamforming perceived by the user in the tth time slot, Indicates drone base station iAdjustment association information with the user in the tth time slot, Indicates drone base station i Whether to perceive information in the tth time slot.

[0033] (5) Reward: The reward of each agent takes into account two optimization objectives, namely, maximizing the user communication throughput and minimizing the flight energy consumption of the drone base station.

[0034] Therefore, after the basic elements such as agent, state, observation, action, and reward are modeled, a reinforcement learning model that can be used to output the flight state adjustment action of the drone base station is obtained through pre-training.

[0035] Specifically, the user's location information at the current moment is sensed through the built-in radar.

[0036] Furthermore, based on the user location information, the parameter configuration of the drone base station and the user is determined. Specifically, the parameter configuration of the drone base station and the user is determined according to the user location information using a semi-positive definite method and a game theory algorithm.

[0037] Furthermore, the current position information and current horizontal speed of the drone base station can be obtained through relevant sensors.

[0038] Furthermore, the parameter configuration of the drone base station and the user, the current position information and the current horizontal speed of the drone base station are determined as the first current state information of the drone base station.

[0039] Furthermore, through the above method, each drone base station in the drone base station cluster can determine its own status information, and therefore, can obtain the second current status information of the neighboring drone base station, wherein the neighboring drone base station refers to other drone base stations other than the drone base station within the preset observation range of the drone base station, and the preset observation range can be set according to actual conditions.

[0040] Further, the first current state information of the drone base station and the second current state information of the neighboring drone base station are determined as the observation information of the drone base station; Furthermore, the observation information of the UAV base station is input into the pre-trained reinforcement learning model to obtain the new flight state output by the reinforcement learning model; wherein the new flight state includes the moving speed and yaw angle of the UAV base station. Then, the position of the UAV base station and the perceived user position are input into the semi-positive definite and game algorithms to obtain the parameter configuration of each UAV base station, including the first target transmit beamforming for user communication, the second target transmit beamforming for user perception, the adjustment association information with the user, and whether the information is perceived.

[0041] Furthermore, the position is adjusted according to the new flight status.

[0042] Therefore, each drone base station can autonomously control the flight status, with lower latency than centralized control. Drone base stations can collaborate with each other to serve ground users together.

[0043] The method for deploying the location of a drone base station provided by the present invention is that the drone base station first senses the current user location information when there is no network side providing the location distribution information of global users, determines the parameter configuration of the drone base station and the user according to the user location information, determines its own state information by using the parameter configuration, and simultaneously obtains the state information of other drone base stations within a preset observation range, determines its own observation information by using its own state information and the state information of surrounding drone base stations, and further combines with a reinforcement learning model to quickly make a decision to update the motion state, so as to achieve that during the real-time movement of the user, the drone base station can quickly adjust the flight state to get closer to the user, and provide the user with high-quality continuous communication services.

[0044] In some embodiments, each drone base station in a deployed drone base station cluster is subject to flight constraints during flight, wherein the flight constraints specifically include a first constraint, a second constraint, a third constraint, a fourth constraint, a fifth constraint, a sixth constraint, a seventh constraint, and an eighth constraint.

[0045] The first constraint C1: , T, which means the yaw angle of the drone base station is within the range.

[0046] The second constraint C2: , T, which means the horizontal speed of the drone base station is within the range.

[0047] The third constraint C3: , T, U, , where i and j both represent the drone base stations in the drone base station cluster U, is a safety distance constant, which means that the moving distance between drone base station i and drone base station j needs to be greater than or equal to the safety distance constant.

[0048] The fourth constraint C4: , T, U, which means the distance the drone base station moves horizontally. within the range.

[0049] The fifth constraint C5: , T, U, which means the distance the drone base station moves horizontally. within the range.

[0050] The sixth constraint C6: , U, ,in is a safety speed constant, which means that the difference in the horizontal speed of flight between drone base station i and drone base station j needs to be less than or equal to the safety speed constant.

[0051] The seventh constraint C7: , T, U, where It is the transmission beamforming of the drone base station to the user communication. It is the transmit beamforming perceived by the drone base station to the user. Indicates whether it is perceived (0 or 1), is the maximum total power, which means that the total power of communication and perception of the drone base station needs to be limited within the maximum total power.

[0052] The eighth constraint C8: , T, U, where is the elevation angle of the drone base station, Indicates whether it is perceived (0 or 1), is a limiting constant, which means that the drone base station perception indicator is limited to a constant.

[0053] Therefore, it can be understood that the first constraint is used to limit the horizontal direction of the drone base station; the second constraint is used to limit the horizontal speed of the drone base station; the third constraint is used to ensure that the drone base stations maintain a safe distance and do not get too close. When the distance between one drone base station and another drone base station during movement is less than the safe distance, it is considered that a collision has occurred; the fourth constraint is used to limit the distance that the drone base station moves in the horizontal direction; the fifth constraint is used to limit the distance that the drone base station moves in the vertical direction; the sixth constraint is used to ensure the stability of flight between drone base stations and reduce the possibility of collision between drone base stations; the seventh constraint is used to limit the total power of communication and perception of the drone base station; the eighth constraint is used to constrain the perception indicator to ensure the effect of the drone base station perception.

[0054] The embodiments of the present invention improve the safety, stability and efficiency of the UAV base station cluster during flight by limiting the flight constraints, so that it can better cope with complex tasks and changing environments, ensure the smooth completion of tasks, and provide users with high-quality continuous communication services.

[0055] In some embodiments, the reinforcement learning model is obtained by pre-training in the following manner: Initialize the action network and the judgment network; Get the current state sample and current observation sample of the drone base station sample; Based on the current observation sample, output a target action through the action network so that the drone base station sample performs the target action; After the drone base station sample performs the target action, a current reward is obtained through observation by the evaluation network; After the drone base station sample performs the target action, obtaining a next state sample and a next observation sample of the drone base station sample; Generate a current trajectory set sample based on the current state information, the current observation sample, the target action, the current reward, the next state sample and the next observation sample; Calculating an advantage estimate of the current trajectory set sample; generating an experience sample based on the current trajectory set sample and the advantage estimate; Updating the next state sample to the current state sample, and updating the next observation sample to the current observation sample, iteratively executing the step of outputting the target action through the action network based on the current observation sample, until a preset number of iterations is reached, obtaining multiple experience samples, and storing the multiple experience samples in a replay buffer; A fixed number of experience samples are randomly extracted from the replay buffer, and the action network and the judgment network are updated to obtain the pre-trained reinforcement learning model.

[0056] Specifically, initialize the replay buffer D, learning rate α and maximum number of iterations G, and set the action network of all agents ={ ,…, } and judgment network φ={φ1,…,φn}, each agent has a corresponding action network and judgment network.

[0057] Traversing the outer loop : Reset the environment and initialize the position and flight state of the drone base station sample, and the trajectory set τ = []; Traversing the loop t=1,…,T: Use the shortest distance algorithm to solve the association problem between drone base station samples and users; Use semi-positive definite algorithm to obtain the communication and perception parameter configuration of the drone base station sample; Get the current state sample and current observation sample of the drone base station sample; Based on the current observation sample, the target action is output through the action network, and the drone base station sample performs position adjustment according to the target action; After the drone base station sample performs the target action, the current reward is obtained through the judgment network observation; After the drone base station sample performs the target action, the next state sample and the next observation sample of the drone base station sample are also obtained; Based on the current state information, current observation sample, target action, current reward, next state sample and next observation sample, the current trajectory set sample is generated, which is expressed as [ ], added to the trajectory set τ; End loop t; Compute the advantage estimate for the current trajectory sample: , this advantage estimate is combined with the λ-return method, which can balance the variance and bias in reinforcement learning, improve the estimation efficiency of the policy gradient, and make the learning process more stable and efficient; Generate experience samples based on the current trajectory set sample and advantage estimate, and store them in the replay buffer; The next state sample is updated as the current state sample, and the next observation sample is updated as the current observation sample, and the loop g=1, ..., G is traversed, and the step of outputting the target action through the action network based on the current observation sample is iteratively executed, and finally a plurality of experience samples are obtained, and the plurality of experience samples are stored in the replay buffer; Randomly extract a fixed number of experience samples from the replay buffer to update the action network and the judgment network; End loop g; Set replaybuffer = ∅; End the loop l .

[0058] Through the above-mentioned pre-training process, the reinforcement learning model of the embodiment of the present invention can effectively learn and optimize the decision-making ability of the drone base station in a dynamic environment. This process not only improves the adaptability and responsiveness of the model to environmental changes, but also optimizes the learning efficiency through the calculation of the advantage estimate, and ultimately realizes the model's efficient decision-making and execution capabilities in complex tasks, significantly improving the autonomous flight and task completion effects of the drone base station cluster.

[0059] In one embodiment, the current reward is obtained in the following manner: Obtaining the flight power consumption of the drone base station sample in the current time slot, and the throughput of the target user communicated by the drone base station sample in the current time slot; Dividing the throughput and the flight power consumption to obtain the energy efficiency of the drone base station sample in the current time slot; Calculate a penalty value when the UAV base station sample violates a flight constraint condition during flight in the current time slot; The energy efficiency is subtracted from the penalty value to obtain the current reward.

[0060] Specifically, in the reinforcement learning process, the aim is to optimize the flight trajectory of the drone base station cluster (i.e. and ) and communication perception parameter configuration, and the flight energy consumption of the drone base station Minimize and maximize user communication throughput Maximization, that is, the energy efficiency of drone base stations Maximize, that is:

[0061] Since the reward for the drone base station sample takes into account the two optimization objectives of maximizing the user communication throughput and minimizing the flight energy consumption of the drone base station, the reward is considered as a vector form between the maximum user communication throughput and the flight energy consumption of the drone base station.

[0062] The current reward calculation formula is as follows:

[0063] in, represents the reward in the current time slot t, represents the throughput of the target user communicating with the drone base station sample in the current time slot t, represents the flight power consumption of the drone base station sample in the current time slot t, It represents the penalty value when the UAV base station sample violates the flight constraint during the flight in the current time slot t. and It is the penalty when the drone base station sample violates the flight constraints. When the drone base station sample flies out of the map range =1, when a collision occurs =1.

[0064] The embodiment of the present invention effectively combines the energy efficiency and penalty mechanism of the drone base station samples to obtain rewards in the pre-training process, ensuring the optimization of energy utilization and compliance with flight constraints in the decision-making process, so that the model can not only optimize energy consumption and communication efficiency, but also effectively maintain flight safety and stability. The comprehensive reward mechanism greatly improves the decision-making quality of the drone base station and promotes efficient and safe flight mission execution.

[0065] In some embodiments, the flight power consumption of the drone base station sample in the current time slot is obtained by: Obtaining the weight of the drone base station sample, and the horizontal speed and vertical speed of the drone base station sample in the current time slot; Based on the weight of the drone base station sample, the horizontal speed and the vertical speed of the drone base station sample in the current time slot, the flight power consumption of the drone base station sample in the current time slot is determined.

[0066] Specifically, it is assumed that the drone base station sample flies at a height H, where H is a positive constant. and Represents the horizontal direction and horizontal speed of the drone base station sample in the current time slot t, satisfying the following constraints ,in is the maximum flight speed that can be achieved in each time slot due to the limited engine power budget.

[0067] The Cartesian coordinate system can be used to simulate the motion of the drone base station sample. Represents the horizontal coordinate of the drone base station sample at time slot t. and , the horizontal coordinate of the drone base station sample at time slot t+1 can be obtained by the following formula

[0068] ; .

[0069] Assume that the drone base station samples are moving at a constant speed in each time slot t Flight, drone base station samples can only be and Move within the rectangular area, that is , .

[0070] When the rotorcraft base station sample is flying, it The total flight power consumption in a time slot t is determined by the following formula:

[0071] in, , , , and is a constant parameter related to aerodynamics, is the weight of the drone base station sample. and are the horizontal and vertical velocities of the UAV base station samples, respectively, which are assumed to remain unchanged in each time slot t.

[0072] Then, when the rotorcraft UAV base station sample is flying, its flight power consumption in the current time slot t can be expressed as: .

[0073] The embodiment of the present invention calculates the flight power consumption by obtaining the weight of the drone base station sample and its horizontal and vertical speeds in the current time slot. This power consumption calculation method based on physical parameters ensures the model's accurate assessment of energy consumption, allowing the drone base station to better optimize energy usage when performing tasks. It also provides important data support for subsequent decision-making, which helps to achieve intelligent flight control and mission execution.

[0074] In some embodiments, the throughput of the target user with which the drone base station sample communicates in the current time slot is obtained in the following manner: Determine a first occurrence probability of a line-of-sight channel between the drone base station sample and the user in the current time slot and a second occurrence probability of a non-line-of-sight channel; Determine a target channel between the drone base station sample and the target user in the current time slot based on the first occurrence probability and the second occurrence probability; Obtaining a transmit beamforming vector for the drone base station sample to communicate with the target user in the current time slot; Determining a signal-to-noise ratio when the target user communicates based on the target channel and the transmit beamforming vector; Based on the signal-to-noise ratio, a throughput of the target user in the current time slot is determined.

[0075] Specifically, the drone base station is equipped with a vertically placed uniform linear array, which can perform radar perception and communication with ground users. The transmission signal of the drone base station i in the current time slot t is:

[0076] in, represents the transmit beamforming vector of the drone base station sample i communicating with user k in the current time slot t, represents the transmit beamforming vector perceived by the drone base station sample i to user o in the current time slot t, and are unrelated communication signals and dedicated sensor signals, Indicates whether the information is perceived.

[0077] The channel between the drone base station sample and the user only considers the line-of-sight channel (LOS), so the first occurrence probability of the line-of-sight channel between the drone base station sample and the user in the current time slot t is:

[0078] Among them, C and D are parameters describing the environment, depending on the propagation environment such as rural, urban or dense urban, is the elevation angle of the drone base station sample i; therefore, the second occurrence probability of the non-line-of-sight channel (NLOS) between the drone base station sample and the user in the current time slot t is: .

[0079] The target channel between the drone base station sample and the target user in the current time slot t is:

[0080] Where d is the antenna spacing, NLoS component is a complex Gaussian random vector with zero mean and unit covariance matrix, Indicates the reference distance = 1 m, and κ < 1 is the additional attenuation factor due to the NLoS condition.

[0081] The signal-to-noise ratio of the target user’s DL communication is:

[0082] in, It refers to the signal-to-noise ratio of the target user in the current time slot t. is the target channel between the drone base station sample and the target user in the current time slot t, is the transmit beamforming vector of the drone base station sample to communicate with the target user in the current time slot t, H is the altitude of the drone base station sample during flight, is a constant parameter related to the signal-to-noise ratio.

[0083] Then, the throughput of the target user communicating with the drone base station sample i in the current time slot t is:

[0084] in, is the throughput of the target user in the current time slot t, is the signal-to-noise ratio of the target user’s DL communication.

[0085] The throughput of the target user for communication of all drone base station samples in the current time slot t is:

[0086] The embodiment of the present invention calculates the occurrence probability of line-of-sight channels and non-line-of-sight channels, determines the target channel, and combines the transmit beamforming vector to determine the signal-to-noise ratio of the target user during communication. Based on the signal-to-noise ratio, the throughput of the target user in the current time slot is obtained, thereby achieving an accurate evaluation of the communication performance between the UAV base station and the user, which can effectively improve the communication capability of the UAV base station in complex environments, and also provides important data support for subsequent decision-making, which helps to realize intelligent flight control and mission execution.

[0087] The drone base station provided by the present invention is described below. The drone base station described below and the drone base station location deployment method described above can be referenced to each other.

[0088] Reference Figure 2 , Figure 2 It is a structural schematic diagram of the UAV base station provided by the present invention.

[0089] The drone base station comprises: The sensing module 210 is used to sense user location information.

[0090] The parameter configuration determination module 220 is used to determine the parameter configuration of the drone base station and the user based on the user location information; the parameter configuration includes the first transmit beamforming for user communication, the second beamforming for user perception, the association information with the user, and whether the information is perceived.

[0091] The first acquisition module 230 is used to acquire current position information and current horizontal speed.

[0092] The state information determination module 240 is used to determine the configuration parameters, the current position information and the current horizontal speed as the first current state information.

[0093] The second acquisition module 250 is used to acquire second current state information of a neighboring drone base station. The neighboring drone base station refers to other drone base stations other than the drone base station within a preset observation range of the drone base station.

[0094] The observation information determination module 260 is used to determine the first current state information and the second current state information as observation information.

[0095] The flight state prediction module 270 is used to input the observation information into the pre-trained reinforcement learning model to obtain a new flight state output by the reinforcement learning model.

[0096] The flight state adjustment module 280 is used to adjust the position according to the new flight state.

[0097] The drone base station provided by the present invention first senses the current user location information when there is no network side providing the location distribution information of global users, determines the parameter configuration of the drone base station and the user according to the user location information, determines its own state information by using the parameter configuration, and simultaneously obtains the state information of other drone base stations within a preset observation range, determines its own observation information by using its own state information and the state information of surrounding drone base stations, and further combines with the reinforcement learning model to quickly make a decision to update the motion state, so as to achieve that the drone base station can quickly adjust the flight state to get closer to the user during the real-time movement of the user, and provide the user with high-quality continuous communication services.

[0098] Furthermore, the drone base station is also used for: The UAV base station is subject to flight constraints during the flight; the flight constraints include a first constraint, a second constraint, a third constraint, a fourth constraint, a fifth constraint, a sixth constraint, a seventh constraint and an eighth constraint; Among them, the first constraint is used to limit the horizontal direction of the drone base station; the second constraint is used to limit the horizontal speed of the drone base station; the third constraint is used to ensure a safe distance between drone base stations; the fourth constraint is used to limit the distance that the drone base station moves in the horizontal direction; the fifth constraint is used to limit the distance that the drone base station moves in the vertical direction; the sixth constraint is used to ensure the stability of flight between drone base stations; the seventh constraint is used to limit the total power of communication and perception of the drone base station; the eighth constraint is used to ensure the perception effect of the drone base station.

[0099] Furthermore, the drone base station is also used for: Initialize the action network and the judgment network; Get the current state sample and current observation sample of the drone base station sample; Based on the current observation sample, output a target action through the action network so that the drone base station sample performs the target action; After the drone base station sample performs the target action, a current reward is obtained through observation by the evaluation network; After the drone base station sample performs the target action, obtaining a next state sample and a next observation sample of the drone base station sample; Generate a current trajectory set sample based on the current state information, the current observation sample, the target action, the current reward, the next state sample and the next observation sample; Calculating an advantage estimate of the current trajectory set sample; generating an experience sample based on the current trajectory set sample and the advantage estimate; Updating the next state sample to the current state sample, and updating the next observation sample to the current observation sample, iteratively executing the step of outputting the target action through the action network based on the current observation sample, until a preset number of iterations is reached, obtaining multiple experience samples, and storing the multiple experience samples in a replay buffer; A fixed number of experience samples are randomly extracted from the replay buffer, and the action network and the judgment network are updated to obtain the pre-trained reinforcement learning model.

[0100] Furthermore, the drone base station is also used for: Obtaining the flight power consumption of the drone base station sample in the current time slot, and the throughput of the target user communicated by the drone base station sample in the current time slot; Dividing the throughput and the flight power consumption to obtain the energy efficiency of the drone base station sample in the current time slot; Calculate a penalty value when the UAV base station sample violates a flight constraint condition during flight in the current time slot; The energy efficiency is subtracted from the penalty value to obtain the current reward.

[0101] Furthermore, the drone base station is also used for: Obtaining the weight of the drone base station sample, and the horizontal speed and vertical speed of the drone base station sample in the current time slot; Based on the weight of the drone base station sample, the horizontal speed and the vertical speed of the drone base station sample in the current time slot, the flight power consumption of the drone base station sample in the current time slot is determined.

[0102] Furthermore, the drone base station is also used for: Determine a first occurrence probability of a line-of-sight channel between the drone base station sample and the user in the current time slot and a second occurrence probability of a non-line-of-sight channel; Based on the first occurrence probability and the second occurrence probability, determining a target channel between the drone base station sample and the target user in the current time slot; Obtaining a transmit beamforming vector for the drone base station sample to communicate with the target user in the current time slot; Determining a signal-to-noise ratio when the target user communicates based on the target channel and the transmit beamforming vector; Based on the signal-to-noise ratio, a throughput of the target user in the current time slot is determined.

[0103] It should be noted that the drone base station provided by the present invention can execute the drone base station location deployment method described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0104] Figure 3 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 3 As shown, the electronic device may include: a processor 310 , a communications interface 320 , a memory 330 and a communication bus 340 , wherein the processor 310 , the communications interface 320 , and the memory 330 communicate with each other via the communication bus 340 . The processor 310 can call the logic instructions in the memory 330 to execute the drone base station location deployment method, which includes: sensing user location information; based on the user location information, determining the parameter configuration of the drone base station and the user; the parameter configuration includes the first transmit beamforming for user communication, the second beamforming for user perception, the association information with the user, and whether the information is perceived; obtaining the current position information and the current horizontal speed; determining the configuration parameters, the current position information and the current horizontal speed as the first current state information; obtaining the second current state information of the neighboring drone base station; the neighboring drone base station refers to other drone base stations other than the drone base station within the preset observation range of the drone base station; determining the first current state information and the second current state information as observation information; inputting the observation information into the pre-trained reinforcement learning model to obtain a new flight state output by the reinforcement learning model; and adjusting the position according to the new flight state.

[0105] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0106] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the drone base station location deployment method provided in the above embodiments, the method including: sensing user location information; determining parameter configuration of the drone base station and the user based on the user location information; the parameter configuration includes a first transmit beamforming for user communication, a second beamforming for user perception, association information with the user, and whether to perceive information; obtaining current location information and current horizontal speed; determining the configuration parameters, the current location information and the current horizontal speed as first current state information; obtaining second current state information of a neighboring drone base station; the neighboring drone base station refers to other drone base stations other than the drone base station within a preset observation range of the drone base station; determining the first current state information and the second current state information as observation information; inputting the observation information into a pre-trained reinforcement learning model to obtain a new flight state output by the reinforcement learning model; and adjusting the position according to the new flight state.

[0107] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the method for deploying the position of the drone base station provided in the above-mentioned embodiments, the method comprising: sensing user position information; determining parameter configuration of the drone base station and the user based on the user position information; the parameter configuration comprises a first transmit beamforming for user communication, a second beamforming for user perception, association information with the user, and whether the information is perceived; obtaining current position information and a current horizontal speed; determining the configuration parameters, the current position information, and the current horizontal speed as first current state information; obtaining second current state information of a neighboring drone base station; the neighboring drone base station refers to other drone base stations other than the drone base station within a preset observation range of the drone base station; determining the first current state information and the second current state information as observation information; inputting the observation information into a pre-trained reinforcement learning model to obtain a new flight state output by the reinforcement learning model; and adjusting the position according to the new flight state.

[0108] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.

[0109] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for deploying a drone base station, characterized in that: Applied to drone base stations, including: Sense user location information; Based on the user location information, determining the parameter configuration of the drone base station and the user; the parameter configuration includes a first transmit beamforming for user communication, a second beamforming for user perception, association information with the user, and whether to perceive information; Get the current position information and current horizontal speed; determining the configuration parameters, the current position information and the current horizontal speed as first current state information; Acquire second current state information of a neighboring drone base station; the neighboring drone base station refers to other drone base stations other than the drone base station within a preset observation range of the drone base station; determining the first current state information and the second current state information as observation information; Inputting the observation information into a pre-trained reinforcement learning model to obtain a new flight state output by the reinforcement learning model; The position is adjusted according to the new flight status.

2. The method for deploying a drone base station according to claim 1, characterized in that: The UAV base station is subject to flight constraints during the flight; the flight constraints include a first constraint, a second constraint, a third constraint, a fourth constraint, a fifth constraint, a sixth constraint, a seventh constraint and an eighth constraint; Among them, the first constraint is used to limit the horizontal direction of the drone base station; the second constraint is used to limit the horizontal speed of the drone base station; the third constraint is used to ensure that a safe distance is maintained between the drone base stations; the fourth constraint is used to limit the distance that the drone base station moves in the horizontal direction; the fifth constraint is used to limit the distance that the drone base station moves in the vertical direction; the sixth constraint is used to ensure the stability of flight between the drone base stations; the seventh constraint is used to limit the total power of communication and perception of the drone base station; the eighth constraint is used to ensure the perception effect of the drone base station.

3. The method for deploying a drone base station according to claim 1, characterized in that: The reinforcement learning model is obtained by pre-training in the following way: Initialize the action network and the judgment network; Get the current state sample and current observation sample of the drone base station sample; Based on the current observation sample, output a target action through the action network so that the drone base station sample performs the target action; After the drone base station sample performs the target action, a current reward is obtained through observation by the evaluation network; After the drone base station sample performs the target action, obtaining a next state sample and a next observation sample of the drone base station sample; Generate a current trajectory set sample based on the current state information, the current observation sample, the target action, the current reward, the next state sample and the next observation sample; Calculating an advantage estimate of the current trajectory set sample; generating an experience sample based on the current trajectory set sample and the advantage estimate; Updating the next state sample to the current state sample, and updating the next observation sample to the current observation sample, iteratively executing the step of outputting the target action through the action network based on the current observation sample, until a preset number of iterations is reached, obtaining multiple experience samples, and storing the multiple experience samples in a replay buffer; A fixed number of experience samples are randomly extracted from the replay buffer, and the action network and the judgment network are updated to obtain the pre-trained reinforcement learning model.

4. The method for deploying the UAV base station location according to claim 3, characterized in that: The current rewards are obtained in the following way: Obtaining the flight power consumption of the drone base station sample in the current time slot, and the throughput of the target user communicated by the drone base station sample in the current time slot; Dividing the throughput and the flight power consumption to obtain the energy efficiency of the drone base station sample in the current time slot; Calculate a penalty value when the UAV base station sample violates a flight constraint condition during flight in the current time slot; The energy efficiency is subtracted from the penalty value to obtain the current reward.

5. The method for deploying the location of the UAV base station according to claim 4, characterized in that: The flight power consumption of the drone base station sample in the current time slot is obtained in the following way: Obtaining the weight of the drone base station sample, and the horizontal speed and vertical speed of the drone base station sample in the current time slot; Based on the weight of the drone base station sample, the horizontal speed and the vertical speed of the drone base station sample in the current time slot, the flight power consumption of the drone base station sample in the current time slot is determined.

6. The method for deploying the location of the UAV base station according to claim 4, characterized in that: The throughput of the target user that the drone base station sample communicates with in the current time slot is obtained in the following way: Determine a first occurrence probability of a line-of-sight channel between the drone base station sample and the user in the current time slot and a second occurrence probability of a non-line-of-sight channel; Based on the first occurrence probability and the second occurrence probability, determining a target channel between the drone base station sample and the target user in the current time slot; Obtaining a transmit beamforming vector for the drone base station sample to communicate with the target user in the current time slot; Determining a signal-to-noise ratio when the target user communicates based on the target channel and the transmit beamforming vector; Based on the signal-to-noise ratio, a throughput of the target user in the current time slot is determined.

7. A drone base station, characterized in that: include: A sensing module, used to sense user location information; A parameter configuration determination module, used to determine the parameter configuration of the drone base station and the user based on the user location information; the parameter configuration includes a first transmit beamforming for user communication, a second beamforming for user perception, association information with the user, and whether to perceive information; A first acquisition module, used to acquire current position information and current horizontal speed; a state information determination module, configured to determine the configuration parameter, the current position information and the current horizontal speed as first current state information; A second acquisition module is used to acquire second current state information of a neighboring drone base station; The neighboring UAV base station refers to other UAV base stations other than the UAV base station within the preset observation range of the UAV base station; an observation information determination module, configured to determine the first current state information and the second current state information as observation information; A flight state prediction module, used to input the observation information into a pre-trained reinforcement learning model to obtain a new flight state output by the reinforcement learning model; The flight state adjustment module is used to adjust the position according to the new flight state.

8. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the drone base station location deployment method as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, the computer-readable storage medium comprising a non-transitory computer-readable storage medium, having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for deploying the location of a drone base station as described in any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for deploying the location of a drone base station as described in any one of claims 1 to 6 are implemented.

Images