Code cache transmission-oriented unmanned aerial vehicle communication support array planning method and device, and electronic equipment

Abstract

The invention relates to the technical field of unmanned aerial vehicle path planning, and provides an unmanned aerial vehicle communication guarantee array position planning method and device for code cache transmission, and electronic equipment, and the method comprises the steps: obtaining an unmanned aerial vehicle path planning model which takes the maximum probability of obtaining a complete file from an unmanned aerial vehicle cache by a user as a target; constructing by taking an unmanned aerial vehicle rotation angle, an unmanned aerial vehicle moving distance, an unmanned aerial vehicle moving range, a coding parameter, a cache limit, a request constraint and a communication condition as constraint conditions; and solving the unmanned aerial vehicle path planning model to obtain a target path of the unmanned aerial vehicle. According to the method, the device and the electronic equipment provided by the invention, transmission is carried out by using multiple unmanned aerial vehicles while the transmission efficiency is ensured, the fault tolerance of the whole transmission process is improved, and the accuracy of data transmission of the unmanned aerial vehicles is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle path planning, and in particular to a method, device and electronic equipment for planning communication support positions of unmanned aerial vehicles for coded cache transmission. Background Art

[0002] In emergency situations such as floods, earthquakes, and mega-city fires, rescue teams on the ground need a large amount of scene perception data to conduct scene modeling in order to better serve the rescue teams.

[0003] However, due to complex terrain and environmental occlusion, the data requested by users is often difficult to be effectively transmitted. Summary of the invention

[0004] The present invention provides a method, device and electronic equipment for planning communication support positions of unmanned aerial vehicles (UAVs) for coding cache transmission, so as to solve the defect in the prior art that the data requested by users is often difficult to be effectively transmitted due to complex terrain and environmental occlusion.

[0005] The present invention provides a method for planning UAV communication support positions for coding cache transmission, comprising the following steps: Obtaining a drone path planning model, wherein the drone path planning model aims to maximize the probability of a user obtaining a complete file from a drone cache, and is constructed with drone turning angle, drone moving distance, drone moving range, encoding parameters, cache limits, request constraints, and communication conditions as constraints; Solve the UAV path planning model to obtain the target path of the UAV.

[0006] According to a UAV communication support position planning method for coding cache transmission provided by the present invention, the constraints of the UAV path planning model include: The UAV movement distance constraint condition is used to constrain the movement distance range of the UAV, and the position constraint condition is determined based on the maximum distance that the UAV can move in a single time period; A UAV movement range constraint condition, wherein the UAV movement range constraint condition is used to constrain the movement range of the UAV, and the UAV movement range constraint condition is determined based on the maximum distance that the UAV can move in the horizontal and vertical directions respectively; A cache limit constraint condition, wherein the cache limit constraint condition constrains the size of cache content slices based on the cache capacity of the drone; Request constraints, where the request constraints are used to constrain users from initiating requests for files; Communication condition constraints, where the communication condition constraints are used to constrain whether the user can eventually obtain the slices requested by the user from the drone.

[0007] According to a UAV communication support array planning method for coding cache transmission provided by the present invention, solving the UAV path planning model to obtain the target path of the UAV includes: The drone path planning, the drone-user matching situation and the drone group cache content selection in the drone path planning model are respectively converted into Markov decision processes, and each Markov decision process is jointly solved to obtain the target path.

[0008] According to a method for planning a UAV communication support position for coding cache transmission provided by the present invention, the first state space in the Markov decision corresponding to the UAV path planning is determined based on the path planning state information; the path planning state information is determined based on the position of the UAV and the relative positions of other UAVs in the UAV group in which the UAV is located; The first action space in the Markov decision corresponding to the UAV path planning is determined based on the moving distance and moving direction of the UAV; The first reward in the Markov decision corresponding to the drone path planning is determined based on the user request hit rate, the corresponding delay when collecting drone transmission, drone crossing the boundary and drone collision penalties.

[0009] According to a method for planning a UAV communication support position for coding cache transmission provided by the present invention, the second state space in the Markov decision corresponding to the matching situation between the UAV and the user is determined based on the UAV position, user request information and the cache situation of the UAV group; The second action space in the Markov decision corresponding to the matching situation between the drone and the user outputs the association situation between the user and the drone; The second reward in the Markov decision corresponding to the drone and user matching situation is determined based on the user request hit rate, the corresponding delay when transmitting through the acquisition drone, and the penalty for the user drone service quantity being in an unreasonable range.

[0010] According to a method for planning UAV communication support positions for coding cache transmission provided by the present invention, the third state space in the Markov decision corresponding to the selection of cache content of the UAV group is determined based on the cache status of the file of the current UAV group and the request status of the user in the time period; The drone swarm cache content selects the third action space output file sequence number in the corresponding Markov decision as the file that needs to be cached by the drone swarm; The third reward in the Markov decision corresponding to the selection of the cache content of the drone swarm is determined based on the user request hit rate, the corresponding delay when transmitted by the collection drone, and the request tendency for the current recommended file in the area.

[0011] According to a UAV communication support position planning method for coding cache transmission provided by the present invention, the request tendency for the current recommended file in the area is determined based on the correlation between the file and the feature, and the tendency of the specific terrain.

[0012] According to a method for planning a UAV communication support position for coding cache transmission provided by the present invention, the Markov decision processes are jointly solved to obtain the target path, including: Based on the joint solution model, each Markov decision process is jointly solved to obtain the target path; The joint solution model is obtained by replacing the multi-layer perceptrons in the actor network and the critic network in the MAPPO model with Transformer models respectively.

[0013] The present invention also provides a UAV communication support position planning device for coding cache transmission, comprising the following units: An acquisition unit is used to acquire a drone path planning model, wherein the drone path planning model is constructed with the goal of maximizing the probability of a user acquiring a complete file from a drone cache and with drone turning angle, drone moving distance, drone moving range, encoding parameters, cache limits, request constraints, and communication conditions as constraints; The solving unit is used to solve the UAV path planning model to obtain the target path of the UAV.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements any of the above-described methods for planning communication support positions for coding cache transmission.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for planning the communication support position of an unmanned aerial vehicle for coding cache transmission as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned methods for planning communication support positions for coding cache transmission.

[0017] The method, device and electronic device for planning the communication support position of unmanned aerial vehicles for coding cache transmission provided by the present invention obtain a path planning model for unmanned aerial vehicles. The path planning model for unmanned aerial vehicles aims to maximize the probability of users obtaining complete files from the cache of unmanned aerial vehicles, and is constructed with the turning angle of unmanned aerial vehicles, the moving distance of unmanned aerial vehicles, the moving range of unmanned aerial vehicles, coding parameters, cache restrictions, request constraints and communication conditions as constraints. The path planning model for unmanned aerial vehicles is then solved to obtain the target path of the unmanned aerial vehicle. While ensuring the transmission efficiency, the method utilizes multiple unmanned aerial vehicles for transmission, improves the fault tolerance of the entire transmission process, and further improves the accuracy of unmanned aerial vehicle data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of the method for planning the communication support position of unmanned aerial vehicles for coding cache transmission provided by the present invention.

[0020] Figure 2 It is a schematic diagram of the UAV communication support position planning model for coding cache transmission provided by the present invention.

[0021] Figure 3 It is a schematic diagram of the joint solution model provided by the present invention.

[0022] Figure 4 It is a structural diagram of the Transformer model provided by the present invention.

[0023] Figure 5 It is a comparison chart of cache hit rate and latency performance of the algorithm training provided by the present invention.

[0024] Figure 6 It is a comparison chart of cache hit rates of different algorithms and various encoding parameters provided by the present invention.

[0025] Figure 7 It is a structural schematic diagram of the UAV communication support position planning device for coding cache transmission provided by the present invention.

[0026] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or precedence. It should be understood that the terms used in this way can be interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type.

[0029] Figure 1 FIG. 1 is a flow chart of a method for planning a communication support position of a UAV for coding cache transmission provided by the present invention. Figure 1 As shown, the method includes step 110 and step 120.

[0030] Step 110, obtaining a drone path planning model, wherein the drone path planning model aims to maximize the probability that a user obtains a complete file from a drone cache, and is constructed with drone turning angle, drone moving distance, drone moving range, encoding parameters, cache limits, request constraints, and communication conditions as constraints.

[0031] Specifically, the embodiment of the present invention takes into account the variability of emergency scenarios and proposes a collaborative optimization solution using drone path planning and coding cache resources to ensure the communication hit rate while making the communication process as low-latency as possible. The embodiment of the present invention uses cache coding to transmit content in slices, pre-slices redundant coding and distributed deployment of large-capacity perception data, and introduces redundant coding technology, such as (n, k) Maximum Distance Separable (MDS).

[0032] Specifically, the original content can be encoded into n content fragments through this encoding method and stored at different nodes. The requesting node only needs to obtain any k of the n content fragments to restore the complete content. The multi-UAV transmission mode effectively overcomes the problems of node failure and transmission failure caused by complex physical environment and electromagnetic interference. In the process, the storage strategy of the UAV cache node and the joint optimization of the transmission power during content transmission are very critical. In addition, the UAV that supports caching can also dynamically cache popular content, dynamically replace the cache content according to the user's request changes to improve the UAV cache hit rate, and effectively improve the service quality. At present, many research works have designed flexible content deployment and distribution strategies for UAVs or D2D (Device-to-Device) users. However, due to the complex and dynamic environment, traditional optimization algorithms are difficult to achieve flexible and efficient optimization. To this end, Deep Reinforcement Learning (DRL) is an effective way to solve the above problems. Through the continuous exploration and utilization of the dynamic network environment, the UAV path planning and the UAV user association are jointly optimized. While ensuring transmission efficiency, the use of multiple drones for transmission improves the fault tolerance of the entire transmission process.

[0033] The embodiment of the present invention takes into account the variability of emergency scenarios and proposes a collaborative optimization scheme using drone path planning and coding cache resources to ensure the communication hit rate while making the communication process as low-delay as possible. On the one hand, with the help of cache-assisted communication. In actual scenarios, the command center is often far away from the requesting user, and coupled with unstable factors such as occlusion caused by disasters, the process of direct transmission from the command center to the user may take a lot of time. Therefore, the embodiment of the present invention uses drones to pre-select the perception data with the highest probability of being requested for caching, and when the request arrives, the data is directly transmitted through the drone. The process of transmitting data from the command center is reduced, and this cache-assisted communication mode will greatly shorten the transmission delay. At the same time, with the help of cache coding, the content is transmitted in pieces, and the large-capacity perception data is pre-sliced ​​redundantly encoded and distributedly deployed. After the content is encoded by (n, k) MDS, the requesting node only needs to obtain k of the n content slices to restore the complete content. This redundant coding method effectively overcomes the node failure and transmission failure caused by complex physical environment and electromagnetic interference. The transmission reliability is further guaranteed. At the same time, in the case of sliced ​​transmission, the multi-threaded receiving capability of ground users is effectively utilized to achieve low-delay transmission. On the other hand, the communication hit rate still needs to be guaranteed while considering low-latency transmission. The embodiment of the present invention further introduces the high-altitude characteristics and flexibility of drones to expand communication coverage. Under comprehensive consideration of the link quality between nodes, cache space size and other constraints, the drone position is planned according to factors such as user location information to achieve the best communication hit rate.

[0034] The embodiment of the present invention further introduces the high altitude characteristics and flexibility of drones to expand communication coverage. Taking into account the limitations of inter-node link quality, cache space size, etc., the drone position is planned according to factors such as user location information to achieve the best communication hit rate.

[0035] Figure 2 Schematic diagram of the UAV path planning system model for coding cache transmission provided by the present invention, such as Figure 2 As shown in Figure 1, the entire system consists of a collection drone group, a transmission drone, and a requesting user. The transmission drone is represented by , the requesting user set is represented as . Transmission drones are used throughout the system to respond to user requests to achieve lower transmission latency. In each time period, the drone adjusts its own status and responds to user requests. In the system, the collection drone swarm encodes and segments the collected perception data and sends it to the transmission drone. The transmission drone then responds to the user's request. Each requesting user in the system corresponds to an emergency rescue team. The drone sends the request to the cluster head of the ground user, and the cluster head of the ground user shares the data to the entire current team.

[0036] In the whole process, the embodiment of the present invention introduces the concept of cache erasure code, MDS ( m,n ) Encoding guidelines A full-size content will be encoded as m The nodes requesting the file need to be at least n The content can only be restored after the nodes that provide the content are connected and the file fragments are successfully obtained. Traditional erasure codes are more commonly used in distributed storage scenarios, but with the improvement of computer performance and encoding strategies, it is now possible to achieve fast encoding and decoding operations on files. Therefore, the embodiment of the present invention introduces encoding into the cache technology, and pre-slices large-size files for redundant encoding and distributed deployment. The receiving node only needs to successfully receive part of the shard data to decode and restore the original file content, thereby effectively reducing the transmission delay and overcoming the problems of node failure and transmission failure caused by complex physical environments and electromagnetic interference. At the same time, the size of a single shard after sharding can be effectively reduced, which can greatly alleviate the various problems caused by the large-size file transmission process.

[0037] Drone mobility model: In the entire request process, the embodiment of the present invention assumes that the user's ground position is relatively fixed, and the drone flies at a fixed altitude to provide services to the user. Therefore, we can use a 2D plane to represent the position of each drone or user. At the same time, in the entire process, we divide the time into multiple time periods as , drones h In time t The position can be expressed as , User u The position can be expressed as During the entire flight process, every two drones maintain a relatively safe distance during flight, and the entire flight process is always limited to the area. × For each time period t , the drone h The moving direction is described as , the drone h The moving distance is described as ,in Represents the maximum distance that the drone can move in a single time period. Through the above expression, we can infer the next time period position of the drone when the drone’s time position is known as as well as .

[0038] Drone Association Model: In the embodiments of the present invention, it is defined Represented as user u Whether to choose a drone h As the target node for providing services. And define Indicates whether the current area allows the drone swarm to cache content f The fragment content. At time t , the drone decides whether to fly to the user h The following constraints need to be met when sending fragments.

[0039] That is, the drone group in the current area caches the user h When the user selects drone h as the node for its service, drone h Start to users u Send file fragments.

[0040] Transmission Model: Embodiments of the present invention define Represented as user u， Can it eventually be from drones? h Get the shard you requested.

[0041] in, Indicates that user u needs to meet three conditions to successfully obtain the shard from drone h, namely, the drone swarm caches the user hThe user selects drone h as the node it serves, and the drone is able to successfully transfer the fragments.

[0042] Embodiments of the present invention define Indicates whether the user can request a complete file. N When there are shards, At this time, the request of user u in time period t can be satisfied. ,on the contrary .when That is, when the user cannot obtain enough shards from the drone cache, the user obtains the file by accessing the collection drone swarm.

[0043] Embodiments of the present invention define Represents the total number of requests issued by all users at all times. Taking maximizing the probability of users obtaining complete files from the drone cache as the objective function, the above problem can be modeled as: Among them, the above formula is the UAV path planning model, (1)-(5) are value constraints, (6) is the UAV turning angle constraint, (7) is the UAV moving distance constraint, (8)-(9) are the UAV moving range constraints, (10) is the encoding parameter constraint, (11) is the cache limit constraint, (12) is the request constraint, and (13) is the communication condition constraint.

[0044] That is, the UAV movement distance constraint is used to constrain the UAV's movement distance range, and the position constraint is determined based on the maximum distance that the UAV can move in a single time period.

[0045] The UAV movement range constraint condition is used to constrain the movement range of the UAV. The UAV movement range constraint condition is determined based on the maximum distance that the UAV can move in the horizontal and vertical directions respectively.

[0046] Therefore, the entire movement trajectory restriction of the drone can be expressed as: The cache limit constraint constrains the size of cache content shards based on the drone's cache capacity.

[0047] Specifically, the embodiment of the present invention considers that there are a total of F The file contents are represented as , and Represents a file f The size of . Indicates the cache capacity of the drone. In the transmission model for cache encoding, the complete file is not directly cached in the drone. Instead, the file is first passed through ( M , N ) encoding and then caching the encoded file segments in the drone. In the embodiment of the present invention, it is considered that the selected cache content segments are distributed to all drones in the area, that is, the number of segments M is equal to the total number of drones H, that is, , after sharding, the size of a single shard is . Indicates drone h A collection of cached content that satisfies the constraints , that is, the total volume of the drone cache content shards is smaller than the drone h The cache capacity.

[0048] Request constraints are used to constrain users from initiating requests for files. Communication condition constraints are used to constrain whether users can eventually obtain the shards they request from the drone.

[0049] Step 120, solving the UAV path planning model to obtain the target path of the UAV.

[0050] Specifically, after obtaining the UAV path planning model, the UAV path planning model can be solved to obtain the target path of the UAV.

[0051] Specifically, the drone path planning, drone and user matching, and drone swarm cache content selection in the drone planning model can be modeled separately and finally solved jointly.

[0052] The method provided by the embodiment of the present invention obtains a drone path planning model. The drone path planning model aims to maximize the probability of a user obtaining a complete file from the drone cache, and is constructed with the drone turning angle, drone moving distance, drone moving range, encoding parameters, cache restrictions, request constraints and communication conditions as constraints, and then solves the drone path planning model to obtain the target path of the drone. While ensuring transmission efficiency, this method uses multiple drones for transmission, improves the fault tolerance of the entire transmission process, and further improves the accuracy of drone data transmission.

[0053] Based on the above embodiment, step 120 includes: Step 121, converting the drone path planning, the drone-user matching situation and the drone group cache content selection in the drone path planning model into Markov decision processes respectively, and jointly solving each Markov decision process to obtain the target path.

[0054] Specifically, the drone path planning, drone-user matching, and drone swarm cache content selection in the drone path planning model are converted into Markov decision processes, respectively, with the goal of optimizing the drone transmission path, user selection, and selecting cache content with a higher hit rate based on user request preferences. The details are as follows: The UAV path planning problem can be defined as an MDP (Markov Decision Process), usually represented by a five-tuple To represent. Among them, is the state set; is a collection of actions; : is the state transition probability function; : is the reward function; is a discount factor, a coefficient used to measure the importance of future rewards.

[0055] Among them, the first state space in the Markov decision corresponding to the UAV path planning is determined based on the path planning state information, and the path planning state information is determined based on the position of the UAV and the relative positions of other UAVs in the UAV group where the UAV is located.

[0056] For example, for drones , the embodiment of the present invention adopts Indicates its location information. Indicates drone The relative position of other drones in the drone group and the user's location information The embodiment of the present invention uses the current drone position and the relative positions of other drones in the drone group as path planning state information, which is specifically expressed as: The first action space in the Markov decision corresponding to the UAV path planning is determined based on the moving distance and moving direction of the UAV.

[0057] For example, with Indicates the moving distance of the drone h, using represents the moving direction of the drone h, and the specific action space is expressed as .

[0058] The first reward in the Markov decision corresponding to the drone path planning is determined based on the user request hit rate, the corresponding delay when collecting drone transmission, drone crossing the boundary, and drone collision penalties.

[0059] For example, the first reward This is the objective function in system modeling. Currently, penalties are imposed based on whether the drone position exceeds the boundary or is too close to other drones. Define user request hit rate : in, Defined as drone crossing the boundary and drone collision penalty, To avoid the delay caused by the transmission of the drone, , , Represent the reward coefficient and the penalty coefficient respectively. Represents the total number of requests in time period t, Indicates whether the user can request the complete file.

[0060] Among them, by collecting the corresponding delay when the drone is transmitted The steps to determine are as follows: Based on the flight altitude characteristics of drones and the influence of complex environments, the transmission model is often based on line-of-sight wireless transmission (LOS) and non-line-of-sight wireless transmission (NLOS). h and users u The probability of a LOS link between them is: in, a,b are constants that depend on the environment, and Indicates at time t Time User u and drones h The straight-line distance, height Indicates the flight altitude of the drone. h With users u The probability of establishing NLOS is: Among them, the path loss of LoS and NLoS links can be expressed as: in, c is the speed of light, Indicates the carrier frequency, d Indicates the distance between the sender and the receiver. and They represent the average additional loss of LOS link and NLOS link respectively. Their size depends on the environment. Therefore, the embodiment of the present invention considers the average path loss of the air-to-ground channel of LOS link and NLOS link. It is expressed as: The transmitting power of the UAV is Assuming that a drone can serve multiple users, the users at the drone's location need to share bandwidth resources, and the drone's downlink uses OFDM (Orthogonal Frequency Division Multiplexing) access. The users are orthogonal and there is no signal interference between them. t Downlink transmission speed from the drone to the requesting user at the moment for: in, Indicates drone h For users u The allocated bandwidth, the embodiment of the present invention adopts a strategy of evenly allocating bandwidth to service users. Expressed as the power of Gaussian white noise, Represented as a drone h To User u The average path loss.

[0061] When the transmission rate meets the following constraints, the fragment can be successfully transmitted to the user u。

[0062] in, Represents a file f The size of N Expressed as ( M, N ) Minimum number of fragments received in encoding N . and Indicates the maximum hovering time limit of the drone.

[0063] Here we define express h Shard to user u When the above rate constraints are met, The value is 1, otherwise it is 0.

[0064] When the user cannot obtain the fragment within the maximum hovering time limit of the drone, the user needs to obtain the complete content from the collection drone group. t Collect the downlink transmission speed from the drone group to the requesting user at all times for: in, Indicates that the collection drone group is a user u The allocated bandwidth, Indicates the transmission power of the acquisition drone. Indicates the collection of drone groups to users u The average path loss is Represents the latency of obtaining complete content from the collection drone swarm.

[0065] The second state space in the Markov decision corresponding to the matching of drones and users is determined based on the drone position, user request information, and the cache status of the drone group, as follows: For example, for drones , we use Indicates its location information. represents the request status of user u in time period t, Indicates the cache status of the current drone group for file f. We use the drone position, user request information and drone group cache status to represent the second state space of user matching, namely: The second action space in the Markov decision corresponding to the matching between the drone and the user outputs the association between the user and the drone.

[0066] For example, It indicates whether user u chooses drone h as the target node to provide services. The action space matched by the user outputs the association between the user and the drone, that is, the second action space is .

[0067] The second reward in the Markov decision corresponding to the drone-user matching situation is determined based on the user request hit rate, the corresponding delay when transmitting through the acquisition drone, and the penalty for the unreasonable range of the number of user drone services.

[0068] Specifically, since the embodiment of the present invention is based on drone fragmentation for transmission, a single user needs to receive fragments transmitted by multiple drones, so each user's selection of an appropriate number of drones has an important impact on system performance. When the user selects too few drones, the minimum number of fragments N for restoring data cannot be reached, and the user's request cannot be fulfilled. When the user selects too many drones, the user occupies too many drone resources, resulting in a decline in system performance. Therefore, the user can set the ideal maximum fragmentation value according to the actual system environment status. When the number of services actually selected by the user is outside the reasonable range, the embodiment of the present invention adds a second penalty : in, Defined as a penalty for an unreasonable number of drone services provided by a user. , Indicates the user request hit rate, Indicates the corresponding delay when transmitted through the acquisition drone.

[0069] The third state space in the Markov decision corresponding to the selection of the cache content of the drone swarm is determined based on the cache status of the file by the current drone swarm and the request status of the user in the time period.

[0070] Indicates the cache status of the current drone group for file f. Indicates the request state of user u in time period t. The embodiment of the present invention comprehensively uses the drone position, user request information and drone group cache status to represent the third state space of the replacement algorithm, namely: The drone swarm cache content selects the third action space output file number f in the corresponding Markov decision as the file that needs to be cached by the drone swarm.

[0071] The third reward in the Markov decision corresponding to the selection of the cache content of the drone swarm is determined based on the user request hit rate, the corresponding delay when transmitted through the collection drone, and the request tendency for the current recommended file in the area.

[0072] For example, the reward of the replacement algorithm is considered from three aspects in the embodiment of the present invention. First, one aspect is still the objective function of the embodiment of the present invention, that is, the user request hit rate. The second aspect is the probability that all request files sent by users in this round are in the cache content of the drone group, that is, the request cache hit rate regardless of whether the transmission is successful. The third aspect is that the request tendency for the current recommended file f in the region is .

[0073] Then, the third reward is: The request tendency for the current recommended file in the region is determined based on the relevance of the file and the feature, as well as the tendency of the specific landform, as follows: At the beginning of each time period, the user will be in the file collection Select a file to initiate a request, represented as . Indicates user u In time t Initiated a file f Request. Indicates user u No application documents f。 The application documents must meet the constraints , that is, each user only requests a single file in a single time period. Since the perceptual data of the terrain is transmitted in the embodiment of the present invention, the files may have different landform features (for example, forests, mountains, buildings, etc.). The perceptual landform features of each file are described as , Represents a file f and Features The constraints are satisfied. .

[0074] In actual emergency scenarios, disaster outbreaks are often concentrated in specific landforms, such as fires often occur in dense forests and landslides occur in mountainous areas. That is, the emergency team's requests will have a tendency towards specific landforms. The tendency of specific landforms is defined as , where the constraints are satisfied According to the above relationship, we can calculate the file f The request tendency is: in, Represents a file f and Features The relevance of Indicates the tendency of a specific landform.

[0075] It should be noted that in the present invention, the entire drone group is regarded as a whole, and the entire drone group is the target of replacement. The present invention replaces the cache content of the entire drone group according to the tendency of user requests, and selects a target file for replacement recommendation in each time period. The cache hit rate of the entire system is guaranteed by dynamically capturing the user's request status and tendency.

[0076] Based on the above embodiment, the step 121 of jointly solving each Markov decision process to obtain the target path includes: Step 1211, based on the joint solution model, jointly solve each Markov decision process to obtain the target path; The joint solution model is obtained by replacing the multi-layer perceptrons in the actor network and the critic network in the MAPPO model with Transformer models respectively.

[0077] Specifically, based on the joint solution model, each Markov decision process is jointly solved to obtain the target path.

[0078] Proximal Policy Optimization (PPO) is a policy gradient algorithm that uses a simple but effective target mini-batch update function to solve the problem that traditional algorithms are sensitive to step size and difficult to determine a reasonable one. PPO uses the Actor-Critic method of reinforcement learning. The Actor network outputs specific behavior based on the policy π, and the Critic network evaluates the value of the behavior. Evaluate the value of the selected action. We use Represents the strategy of the Actor network, using Represents the evaluation strategy of the Critic network, where and Denote the network parameters of the Actor network and the Critic network respectively. The pruning substitution objective function of PPO is expressed as follows: in, represents the ratio of the new strategy to the old strategy, is a clipping score. It is to avoid over-modification of the value. The generalized advantage estimator (GAE) is defined as: The parameters of the Critic network are It means that the training is done with a gradient descent loss function, which is mainly expressed as in, .

[0079] The original work of single-agent PPO proposed using entropy to enhance the overall objective function reward To promote exploration. In summary, the objective function can be expressed as: in , represents the specific coefficient, Indicates that when the status is The entropy of the strategy.

[0080] Multi-agent reinforcement learning can provide a distributed perspective for agents, realize distributed execution of each agent, and jointly optimize multiple agents to achieve the highest reward. The embodiment of the present invention adopts MAPPO (Multi-Agent Proximal Policy Optimization) for solution. In MAPPO, each drone is regarded as an agent, and cache replacement and user association of two agents are additionally added. These agents execute the PPO algorithm separately and evaluate the reward separately. MAPPO adopts a decentralized execution framework. Each agent has two stages: 1) centralized training, 2) decentralized execution.

[0081] The training phase is offline, while exploration is used to explore the optimal strategy. In the execution phase, it only requires forward propagation, without the need for a random exploration process. Next, we take the agent as an example to illustrate how to centrally train the MAPPO model and execute the learning model in a decentralized approach.

[0082] During the centralized training phase, the Critic network acts as a central coordinator and computes a function of the centralized action value based on the global state that includes the actions and observations of all agents. This centralized The function evaluates the actor's actions globally and guides it to choose better actions. The Critic Network then updates its parameters by minimizing the loss .

[0083] in, , in , are the network parameters of the Critic network.

[0084] And the Actor network is a function of the centralized action value Update its network parameters The updated direction is expressed as .

[0085] Figure 3 is a schematic diagram of the joint solution model provided by the present invention, Figure 4 is a schematic diagram of the structure of the Transformer model provided by the present invention, such as Figure 3 , Figure 4As shown, the embodiment of the present invention proposes the TMAPPO algorithm based on the MAPPO algorithm and the Transformer structure in the large model technology. The Transformer structure is added to the actor network and the critic network structure in MAPPO. The Transformer network is an important component of today's large models. The Transformer structure has a self-attention mechanism for learning the contextual association between inputs.

[0086] In the scenario of the embodiment of the present invention, since the critic network needs to receive observation information from multiple agents, and the data observed by multiple agents in multi-agent reinforcement learning also have certain connections, the structure of the Transformer is added to the critic network. Compared with MLP (Multilayer Perceptron), Transformer can more effectively associate between agents and accelerate the convergence speed of the algorithm. At the same time, compared with a single self-attention network, the multi-head attention mechanism adopted by Transformer, in training, each set is used to embed the input into a different representation subspace, which expands the model's ability to focus on different positions and improves the ability to capture related information. In the embodiment of the present invention, we further utilize the ability of Transformer and innovatively integrate Transformer into the actor network. Since the embodiment of the present invention uses multiple drones to transmit files collaboratively. Each agent also needs the location information of other drones when planning the path in order to obtain sufficient information for collaborative optimization. Therefore, in the scenario of the embodiment of the present invention, the input in the actor network also has the collaborative location information of multiple agents. Incorporating the Transformer network into this input information can also enable the ability to quickly capture contextual information, output behavior based on information from multiple drones, and further improve the performance of the algorithm.

[0087] The algorithm flow of the embodiment of the present invention is as follows: The performance of the present invention is verified by experimental simulation results. The deep reinforcement learning process is based on PyTorch. In the simulation setting, points are randomly scattered within a range of 500m×500m, the number of drone cache nodes is 5, and the number of request nodes is 20. The total number of contents in the content set is 50, and the storage capacity of a single drone cache node is 1g.

[0088] Figure 5 This is a comparison chart of cache hit rate and latency performance of the algorithm training provided by the present invention, such as Figure 5 As shown, Figure 5(a) shows the final change of cache hit rate of the algorithm of the present invention in this environment. The algorithm of the present invention can achieve balance and effectively converge to the global optimum, which shows that through multi-agent algorithm learning, multiple agents can cooperate to make the best decision, achieve common goals, and finally achieve an ideal hit rate. Figure 5 (b) The training conditions of the TMAPPO algorithm proposed in the embodiment of the present invention are compared with the MAPPO-Critic algorithm that only adds the critic network to the transformer network and the original MAPPO algorithm. It can be seen that the MAPPO algorithm proposed in this article can converge to the global optimum faster, which indicates that the introduced transformer network effectively captures the information between multiple agents during the training process to accelerate the convergence of the algorithm. Figure 5 (c) The average delay of the present invention for fragment transmission and the average delay of the direct transmission of the entire file are compared. From the results, it can be seen that the fragment scheduling method can obtain a lower average delay than the traditional method.

[0089] Figure 6 This is a comparison chart of cache hit rates of different algorithms and various encoding parameters provided by the present invention, such as Figure 6 As shown, Figure 6 (a) compares the cache request response rate obtained by the planning algorithm in this paper with the random flight of the drone, the flight of the drone around the map, and the use of the planned path in this paper without sharding. The horizontal axis represents the maximum hovering time allowed by the drone. It can be seen that the path planning algorithm in this paper has brought a significant improvement to the overall cache request response rate. At the same time, by comparing the results of the whole packet transmission and the sharded transmission, it can be found that the lower the hovering time, the more advantages of the sharded transmission can be reflected. This is because it is difficult to complete the transmission of the whole packet when the hovering time is low. At this time, the sharded transmission mode of this paper is used to reduce the file size of a single transmission, and the concurrent receiving capability of the user can be used to more significantly improve the cache request response rate. Figure 6 (b) compares the effect of the encoding parameter K on the transmission hit rate under different numbers of users. From the results, it can be seen that when the number of drones is 5, the (5,3) encoding can achieve better results. In order to compare the disaster recovery characteristics of the present invention, we compared the transmission of the entire file under different encoding parameters and the decrease in the response hit rate of the fragmented transmission in the present invention when a single drone is damaged. It can be seen from 6(c) that the decrease in the hit rate of the fragmented transmission is lower when the drone is damaged, which proves that the encoding fragmentation strategy of this paper brings better disaster recovery performance to the system, and the disaster recovery effect will be more significant as the redundancy increases.

[0090] The method provided in the embodiment of the present invention is based on the MAPPO algorithm and combines the Transformer structure in the large model technology to propose the TMAPPO algorithm. The Transformer structure is added to the actor network and the critic network structure in MAPPO, and the Transformer self-attention mechanism is used to learn the contextual association between inputs, that is, the association of multiple drones working together. Under this method, multiple drones can cooperate in transmission work, effectively improve the cache hit situation of large files, and accelerate the convergence of the algorithm.

[0091] The following is a description of the UAV communication support position planning device for coding cache transmission provided by the present invention. The UAV communication support position planning device for coding cache transmission described below and the UAV communication support position planning method for coding cache transmission described above can be referenced to each other.

[0092] Based on any of the above embodiments, the present invention provides a UAV communication support position planning device for coding cache transmission. Figure 7 : is a schematic diagram of the structure of the UAV communication support position planning device for coding cache transmission provided by the present invention, such as Figure 7 As shown, the device comprises: An acquisition unit 710 is used to acquire a drone path planning model, wherein the drone path planning model is constructed with the drone turning angle, drone moving distance, drone moving range, encoding parameters, cache limit, request constraint and communication condition as constraints, with the goal of maximizing the probability of a user obtaining a complete file from the drone cache. The solving unit 720 is used to solve the UAV path planning model to obtain the target path of the UAV.

[0093] The device provided by the embodiment of the present invention obtains a drone path planning model. The drone path planning model aims to maximize the probability of a user obtaining a complete file from the drone cache, and is constructed with the drone turning angle, drone moving distance, drone moving range, encoding parameters, cache restrictions, request constraints and communication conditions as constraints, and then solves the drone path planning model to obtain the target path of the drone. While ensuring transmission efficiency, this method uses multiple drones for transmission, improves the fault tolerance of the entire transmission process, and further improves the accuracy of drone data transmission.

[0094] Based on any of the above embodiments, the constraints of the drone path planning model include: The UAV movement distance constraint condition is used to constrain the movement distance range of the UAV, and the position constraint condition is determined based on the maximum distance that the UAV can move in a single time period; A UAV movement range constraint condition, wherein the UAV movement range constraint condition is used to constrain the movement range of the UAV, and the UAV movement range constraint condition is determined based on the maximum distance that the UAV can move in the horizontal and vertical directions respectively; A cache limit constraint condition, wherein the cache limit constraint condition constrains the size of cache content slices based on the cache capacity of the drone; Request constraints, where the request constraints are used to constrain users from initiating requests for files; Communication condition constraints, where the communication condition constraints are used to constrain whether the user can eventually obtain the slices requested by the user from the drone.

[0095] Based on any of the above embodiments, the solving unit 720 is specifically used for: The joint solving unit is used to convert the drone path planning, the drone-user matching situation and the drone group cache content selection in the drone path planning model into Markov decision processes respectively, and jointly solve each Markov decision process to obtain the target path.

[0096] Based on any of the above embodiments, the first state space in the Markov decision corresponding to the drone path planning is determined based on the path planning state information; the path planning state information is determined based on the drone position and the relative positions of other drones in the drone group in which the drone is located; The first action space in the Markov decision corresponding to the UAV path planning is determined based on the moving distance and moving direction of the UAV; The first reward in the Markov decision corresponding to the drone path planning is determined based on the user request hit rate, the corresponding delay when collecting drone transmission, drone crossing the boundary and drone collision penalties.

[0097] Based on any of the above embodiments, the second state space in the Markov decision corresponding to the matching situation between the drone and the user is determined based on the drone position, the user request information and the cache situation of the drone group; The second action space in the Markov decision corresponding to the matching situation between the drone and the user outputs the association situation between the user and the drone; The second reward in the Markov decision corresponding to the drone and user matching situation is determined based on the user request hit rate, the corresponding delay when transmitting through the acquisition drone, and the penalty for the user drone service quantity being in an unreasonable range.

[0098] Based on any of the above embodiments, the third state space in the Markov decision corresponding to the drone swarm cache content selection is determined based on the current drone swarm cache state of the file and the user's request state in the time period; The drone swarm cache content selects the third action space output file sequence number in the corresponding Markov decision as the file that needs to be cached by the drone swarm; The third reward in the Markov decision corresponding to the selection of the cache content of the drone swarm is determined based on the user request hit rate, the corresponding delay when transmitted by the collection drone, and the request tendency for the current recommended file in the area.

[0099] Based on any of the above embodiments, the request tendency for the current recommended file in the area is determined based on the relevance between the file and the feature, and the tendency of the specific landform.

[0100] Based on any of the above embodiments, the joint solving unit is specifically used for: Based on the joint solution model, each Markov decision process is jointly solved to obtain the target path; The joint solution model is obtained by replacing the multi-layer perceptrons in the actor network and the critic network in the MAPPO model with Transformer models respectively.

[0101] Figure 8 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the UAV communication guarantee position planning method for coding cache transmission, the method comprising: obtaining a UAV path planning model, the UAV path planning model is constructed with the UAV turning angle, the UAV moving distance, the UAV moving range, the coding parameters, the cache limit, the request constraint and the communication condition as the constraint conditions to maximize the probability of the user obtaining the complete file from the UAV cache; solving the UAV path planning model to obtain the target path of the UAV.

[0102] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0103] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the UAV communication guarantee position planning method for coding cache transmission provided by the above methods, and the method includes: obtaining a UAV path planning model, the UAV path planning model aims to maximize the probability of a user obtaining a complete file from the UAV cache, and is constructed with UAV turning angle, UAV moving distance, UAV moving range, coding parameters, cache restrictions, request constraints and communication conditions as constraints; solving the UAV path planning model to obtain the target path of the UAV.

[0104] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the UAV communication support position planning method for coding cache transmission provided by the above-mentioned methods, the method comprising: obtaining a UAV path planning model, the UAV path planning model is aimed at maximizing the probability of a user obtaining a complete file from the UAV cache, and is constructed with UAV turning angle, UAV moving distance, UAV moving range, coding parameters, cache restrictions, request constraints and communication conditions as constraints; solving the UAV path planning model to obtain the target path of the UAV.

[0105] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0106] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for planning UAV communication support positions for coding cache transmission, characterized in that: include: Obtaining a drone path planning model, wherein the drone path planning model aims to maximize the probability of a user obtaining a complete file from a drone cache, and is constructed with drone turning angle, drone moving distance, drone moving range, encoding parameters, cache limits, request constraints, and communication conditions as constraints; Solve the UAV path planning model to obtain the target path of the UAV.

2. The method for planning UAV communication support positions for coding cache transmission according to claim 1 is characterized in that: The constraints of the UAV path planning model include: The UAV movement distance constraint condition is used to constrain the movement distance range of the UAV, and the position constraint condition is determined based on the maximum distance that the UAV can move in a single time period; A UAV movement range constraint condition, wherein the UAV movement range constraint condition is used to constrain the movement range of the UAV, and the UAV movement range constraint condition is determined based on the maximum distance that the UAV can move in the horizontal and vertical directions respectively; A cache limit constraint condition, wherein the cache limit constraint condition constrains the size of cache content slices based on the cache capacity of the drone; Request constraints, where the request constraints are used to constrain users from initiating requests for files; Communication condition constraints, where the communication condition constraints are used to constrain whether the user can eventually obtain the slices requested by the user from the drone.

3. The method for planning UAV communication support positions for coding cache transmission according to claim 1 is characterized in that: The step of solving the UAV path planning model to obtain the target path of the UAV includes: The drone path planning, the drone-user matching situation and the drone group cache content selection in the drone path planning model are respectively converted into Markov decision processes, and each Markov decision process is jointly solved to obtain the target path.

4. The method for planning UAV communication support positions for coding cache transmission according to claim 3 is characterized in that: The first state space in the Markov decision corresponding to the UAV path planning is determined based on the path planning state information; the path planning state information is determined based on the position of the UAV and the relative positions of other UAVs in the UAV group where the UAV is located; The first action space in the Markov decision corresponding to the UAV path planning is determined based on the moving distance and moving direction of the UAV; The first reward in the Markov decision corresponding to the drone path planning is determined based on the user request hit rate, the corresponding delay when collecting drone transmission, drone crossing the boundary and drone collision penalties.

5. The method for planning UAV communication support positions for coding cache transmission according to claim 3 is characterized in that: The second state space in the Markov decision corresponding to the matching of the drone and the user is determined based on the drone position, the user request information and the drone group cache status; The second action space in the Markov decision corresponding to the matching situation between the drone and the user outputs the association situation between the user and the drone; The second reward in the Markov decision corresponding to the drone and user matching situation is determined based on the user request hit rate, the corresponding delay when transmitting through the acquisition drone, and the penalty for the user drone service quantity being in an unreasonable range.

6. The method for planning UAV communication support positions for coding cache transmission according to claim 3 is characterized in that: The third state space in the Markov decision corresponding to the selection of the cache content of the drone swarm is determined based on the cache status of the file by the current drone swarm and the request status of the user in the time period; The drone swarm cache content selects the third action space output file sequence number in the corresponding Markov decision as the file that needs to be cached by the drone swarm; The third reward in the Markov decision corresponding to the selection of the cache content of the drone swarm is determined based on the user request hit rate, the corresponding delay when transmitted by the collection drone, and the request tendency for the current recommended file in the area.

7. The method for planning UAV communication support positions for coding cache transmission according to claim 6 is characterized in that: The request tendency for the current recommended file in the area is determined based on the relevance of the file and the feature, and the tendency of the specific landform.

8. The method for planning UAV communication support positions for coding cache transmission according to claim 3 is characterized in that: The step of jointly solving each Markov decision process to obtain the target path includes: Based on the joint solution model, each Markov decision process is jointly solved to obtain the target path; The joint solution model is obtained by replacing the multi-layer perceptrons in the actor network and the critic network in the MAPPO model with Transformer models respectively.

9. A UAV communication support position planning device for coding cache transmission, characterized in that: include: An acquisition unit is used to acquire a drone path planning model, wherein the drone path planning model is constructed with the goal of maximizing the probability of a user acquiring a complete file from a drone cache and with drone turning angle, drone moving distance, drone moving range, encoding parameters, cache limits, request constraints, and communication conditions as constraints; The solving unit is used to solve the UAV path planning model to obtain the target path of the UAV.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for planning UAV communication support positions for coding cache transmission according to any one of claims 1 to 8 is implemented.

Images