A path planning method for unmanned aerial vehicle assisted wireless communication system

Abstract

The present invention relates to a path planning method for a UAV-assisted wireless communication system. The method adopts the Nested Monte Carlo Search (NMCS) technique to dynamically plan the flight path of the UAV to adapt to the changing needs of the rescue site. The system provides computing task offloading services for ground users by optimizing, allocating and adjusting computing resources, while considering constraints such as energy consumption. The NMCS algorithm initializes the state and nesting level, sets the initial reward value, and checks in a loop whether the current state is a terminal state. For each legal move, the algorithm calculates the new state and performs random simulation or recursive calls to the NMCS function. The method records and compares the highest rewards in all moves to update the best reward and the selected path.

Description

Technical Field

[0001] The present invention belongs to the field of path planning, and in particular relates to a path planning method for an unmanned aerial vehicle assisted wireless communication system. Background Art

[0002] In disaster response and emergency communication scenarios, drone-assisted wireless systems play a vital role, greatly improving the efficiency and response speed of rescue operations by providing computing task offloading services for ground rescue teams. The application range of such systems is wide, not only limited to the collection and distribution of information, but also includes the optimal allocation and dynamic adjustment of computing resources to adapt to the changing needs of the rescue site. Drones act as mobile computing centers at disaster sites, processing and distributing critical data. This data includes maps, location information, real-time video, etc., which are essential for guiding rescue operations and decision-making. By offloading these computing-intensive tasks, drones help rescue teams quickly obtain processed data, thereby improving the speed and efficiency of rescue operations. Drones minimize energy consumption and extend their service time in the air by optimizing flight paths. This is particularly important at disaster sites because energy supply is often limited and drones need to operate for a long time to provide continuous support. Through intelligent path planning, drones can serve ground users more effectively and ensure the continuity of critical missions.

[0003] In emergency response, the rapid transmission of large amounts of information is crucial. The drone-assisted system maximizes the average throughput of the network while complying with energy and fairness constraints by adjusting the position of the drone and the allocation of computing resources. This optimization ensures that a large amount of rescue information such as location data, rescue actions, and medical data can be transmitted quickly and accurately, thereby improving rescue efficiency. Considering the rapid changes in the situation at the disaster site, the drone's online path planning and resource allocation capabilities show extremely high adaptability. Drones in existing technologies are not able to dynamically adjust according to the real-time movement and demand changes of ground users. How to use path planning algorithms to achieve path optimization for drone-assisted wireless communication systems has become an important research direction in the current industry. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a path planning method for a drone-assisted wireless communication system in response to the above-mentioned deficiencies in the prior art, which can improve the efficiency of rescue while ensuring fairness and complete the computing task offloading service more quickly and accurately.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0006] A method for path planning of an unmanned aerial vehicle assisted wireless communication system comprises the following steps:

[0007] Step 1: Initialize the drone state and the nested level of the nested Monte Carlo search. At this point, set the basic state of the drone, including the initial speed and load the ground rescue team's mission;

[0008] Step 2: Initialize the best reward value and the best path obtained by the drone, and set up an evaluation function to take into account the fairness of the service to ensure that priority is given to the rescue team with a large workload;

[0009] Step 3: In each iteration, check whether the current drone state is in the terminal state, the drone has run out of power, has covered all rescue teams, or has completed all planned flight missions;

[0010] Step 4: If the current drone state is not the terminal state, iterate each legal movement of the drone state, consider the location of each rescue team, the amount of tasks, and the current state of the drone, and plan a reasonable flight path;

[0011] Step 5: For each move, calculate the reward value of the new state, which includes not only the cost of reaching the new location, but also the effect of providing computational offloading services at that location, especially for rescue teams with large workloads;

[0012] Step 6: If the nesting level is 1, perform a random simulation to evaluate which strategy is most effective in the search at the lower level by simulating various possible flight and service scenarios;

[0013] Step 7: If the nesting level is greater than 1, recursively call the nested Monte Carlo search to further refine the search and explore the impact of more levels of decision-making, including how to prioritize rescue teams with large workloads;

[0014] Step 8: In each iteration, record the maximum reward value of all moves and consider how to balance the fairness and efficiency of rescue services through the path selection of the drone;

[0015] Step 9: If the maximum reward value is greater than the current optimal reward value, update the optimal reward value and select the path associated with it. Emphasize the path selection of rescue teams that can improve rescue efficiency while taking into account the large workload;

[0016] Step 10: After the algorithm completes all iterations, it returns the best reward and the best path. This final result will provide the best computational offloading service solution for ground rescue, ensuring the fairness of the service and optimizing the efficiency of resource utilization.

[0017] To optimize the above technical solutions, the specific measures taken also include:

[0018] In the above step 1, the drone state is first initialized, including setting the basic state of the drone, including the initial speed and loading the task of the ground rescue team. At the same time, the nesting level of the nested Monte Carlo search is initialized. This step is very important because the setting of the nesting level will directly affect the accuracy and efficiency of path planning. By reasonably setting the nesting level, the breadth and depth of exploration can be balanced during the search process, so as to better find the optimal path.

[0019] In step 2 above, the system initializes the optimal reward value obtained by the drone and the optimal path selected. In this process, an evaluation function is introduced, which takes into account the fairness of the service and ensures that priority is given to rescue teams with large workloads. Through the establishment of the evaluation function, the system can comprehensively consider the needs of various rescue teams during the path planning process and weigh the fairness and efficiency of the service when selecting the optimal path.

[0020] In step 3 above, the system checks whether the current drone status is in a terminated state in each iteration, which includes determining whether the drone’s battery is exhausted, whether all rescue teams have been covered, or whether all planned flight missions have been completed. The purpose of this step is to ensure that the drone can make timely termination or adjustment decisions during the path planning process to avoid ineffective searches and waste of resources.

[0021] In step 4 above, the system iterates each legal move of the current drone state. In this step, the system takes into account the location of each rescue team, the amount of mission, and the current state of the drone to plan a reasonable flight path. By iterating each legal move, the system can gradually explore possible paths in the search space to find the optimal flight path.

[0022] In step 5 above, the system calculates the reward value of the new state for each move. This reward value not only includes the cost of reaching the new location, but also takes into account the effect of providing computing offloading services at that location, especially for rescue teams with large workloads. By calculating the reward value of the new state, the system can evaluate the potential benefits of each move and optimize the flight path of the drone during the path planning process to maximize service efficiency and ensure fairness of the service.

[0023] In step 6 above, if the nesting level is 1, a random simulation is performed. By simulating various possible flight and service scenarios, the system evaluates which strategy is most effective in the lower-level search. This step helps the system to conduct preliminary exploration in the search space and provide reference and guidance for subsequent path planning.

[0024] In step 7 above, if the nesting level is greater than 1, the system recursively calls the nested Monte Carlo search, which allows the search to be further refined and explore the impact of decisions at more levels, including how to give priority to rescue teams with large workloads. By recursively calling the search algorithm, the system can optimize the path at a deeper level and comprehensively consider more factors to find the optimal flight path.

[0025] In step 8 above, the system records the maximum reward value of all moves in each iteration and considers how to balance the fairness and efficiency of the rescue service through the path selection of the drone. The key to this step is to comprehensively consider the reward value of each move and how to weigh the fairness and efficiency of the service in path selection. By recording the maximum reward value, the system can dynamically adjust the path selection strategy to maximize service efficiency and ensure fairness.

[0026] In the above step 9, if the maximum reward value is greater than the current optimal reward value, the system updates the optimal reward value and selects the path associated with it, focusing on those paths that can improve rescue efficiency while taking into account the rescue teams with large task volumes. The purpose of this step is to ensure that the system selects the optimal path to improve rescue efficiency and optimize resource utilization while ensuring fairness of service.

[0027] In the above step 10, after the algorithm completes all iterations, the system returns the best reward and the best path. This final result will provide the best computational offloading service solution for ground rescue, ensure the fairness of the service and optimize the efficiency of resource utilization. By returning the best reward and the best path, the system can provide a practical and executable path planning solution, providing effective support for the operation of the UAV-assisted wireless communication system.

[0028] The beneficial effects of the present invention are as follows: the system provides computing task offloading services for ground users by optimizing, allocating and adjusting computing resources, while considering constraints such as energy consumption. The NMCS algorithm initializes the state and nesting level, sets the initial reward value, and checks in a loop whether the current state is a terminal state. For each legal move, the algorithm calculates a new state and performs random simulation or recursive calls to the NMCS function. The method records and compares the highest rewards among all moves to update the best reward and the selected path. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is an optional flow chart of a method for path planning of a UAV-assisted wireless communication system according to the present invention;

[0030] Figure 2 A simulation scene diagram of an embodiment of the present invention;

[0031] Figure 3is a comparison diagram of average throughputs of embodiments of the present invention;

[0032] Figure 4 FIG. 4 is a comparison diagram of throughputs at different hovering points according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0034] Although the steps in the present invention are arranged with numbers, they are not used to limit the order of the steps. Unless the order of the steps is clearly stated or the execution of a certain step requires other steps as a basis, the relative order of the steps can be adjusted. It can be understood that the term "and / or" used in this article refers to and covers any and all possible combinations of one or more of the associated listed items.

[0035] A method for path planning of a UAV-assisted wireless communication system can improve the efficiency of rescue while ensuring fairness, and can complete the computing task offloading service more quickly and accurately, such as Figure 1 shown.

[0036] A method for path planning of an unmanned aerial vehicle assisted wireless communication system comprises the following steps:

[0037] Step 1: Initialize the UAV state and the nesting level of the nested Monte Carlo search. At this time, set the basic state of the UAV, such as the initial speed, and load the tasks of the ground rescue team. First, the UAV state is initialized, including setting the basic state of the UAV, including the initial speed, and loading the tasks of the ground rescue team. At the same time, the nesting level of the nested Monte Carlo search is initialized. This step is crucial because the setting of the nesting level will directly affect the accuracy and efficiency of path planning. By reasonably setting the nesting level, the breadth and depth of exploration can be balanced during the search process, so as to better find the optimal path.

[0038] Step 2: Initialize the best reward value and the best path selected by the drone, and set up an evaluation function to take into account the fairness of the service and ensure that priority is given to rescue teams with large workloads. The system initializes the best reward value and the best path selected by the drone. In this process, the evaluation function is introduced. The evaluation function formula is as follows:

[0039]

[0040] In the above formula, Represents the number of offload tasks, where is a binary variable indicating the service status of the rescue team, i.e., whether it is receiving service, and θ represents a predetermined threshold of the number of tasks. If the rescue team has received service before and the subsequent task volume exceeds θ, the rescue team will receive service again;

[0041] This function takes into account the fairness of the service and ensures that priority is given to rescue teams with large workloads. Through the establishment of the evaluation function, the system can comprehensively consider the needs of each rescue team in the path planning process and weigh the fairness and efficiency of the service when selecting the best path.

[0042] Step 3: In each iteration, check whether the current drone status is in a terminated state, whether the drone has run out of power, has covered all rescue teams, or has completed all planned flight missions; The system checks whether the current drone status is in a terminated state in each iteration, which includes determining whether the drone has run out of power, has covered all rescue teams, or has completed all planned flight missions. The purpose of this step is to ensure that the drone can make timely termination or adjustment decisions during the path planning process to avoid ineffective searches and waste of resources.

[0043] Step 4: If the current drone state is not the terminal state, iterate each legal move of the drone state, consider the location of each rescue team, the amount of tasks, and the current state of the drone, and plan a reasonable flight path; The system iterates each legal move of the current drone state. In this step, the system considers the location of each rescue team, the amount of tasks, and the current state of the drone to plan a reasonable flight path. By iterating each legal move, the system can gradually explore possible paths in the search space to find the optimal flight path.

[0044] Step 5: For each move, the reward value of the new state is calculated, which includes not only the cost of reaching the new location, but also the effect of providing computing offloading services at that location, especially for rescue teams with large workloads; The system calculates the reward value of the new state for each move. This reward value includes not only the cost of reaching the new location, but also the effect of providing computing offloading services at that location, especially for rescue teams with large workloads. By calculating the reward value of the new state, the system can evaluate the potential benefits of each move and optimize the flight path of the drone during the path planning process to maximize service efficiency and ensure fairness of the service.

[0045] Step 6: If the nesting level is 1, a random simulation is performed. By simulating various possible flight and service solutions, the system evaluates which strategy is most effective in the search at a lower level. If the nesting level is 1, a random simulation is performed. By simulating various possible flight and service solutions, the system evaluates which strategy is most effective in the search at a lower level. This step helps the system to conduct preliminary exploration in the search space and provide reference and guidance for subsequent path planning.

[0046] Step 7: If the nesting level is greater than 1, the system recursively calls the nested Monte Carlo search, which allows the search to be further refined and explores the impact of decisions at more levels, including how to give priority to rescue teams with large workloads. If the nesting level is greater than 1, the system recursively calls the nested Monte Carlo search, which allows the search to be further refined and explores the impact of decisions at more levels, including how to give priority to rescue teams with large workloads. By recursively calling the search algorithm, the system can optimize the path at a deeper level and comprehensively consider more factors to find the optimal flight path.

[0047] The search algorithm process is as follows:

[0048] Algorithm 2 Applying Simulation-based NMCS for UAV-aidedCommunication Rescue

[0049] 1: selectedTrj ← [], bestReward ← -∞, bestTrj ← []

[0050] 2: while battery E>0 do

[0051] 3: while s is not terminal do

[0052] 4: for each m in legalMoves(s) do

[0053] 5: s' ← Play(s, m)

[0054] 6: if level == 1 then

[0055] 7: (reward, Trj) ← run Playout(s', m)

[0056] 8: else

[0057] 9: (reward, Trj) ← NMCS(s', level - 1)

[0058] 10:highReward ← highest reward of the moves from s

[0059] 11: if highReward>bestReward then

[0060] 12: bestReward ← highReward

[0061] 13: selectedMove ← m associated with highReward

[0062] 14: bestTrj ← Trj associated with highReward

[0063] 15: else

[0064] 16: selectedMove ← first move in bestTrj

[0065] 17:bestTrj ← remove first move from bestTrj

[0066] 18: s ← apply selectedMove on s

[0067] 19:selectedTrj ← append selectedMove to selectedTrj

[0068] 20:

[0069] return (bestReward, selectedTrj)

[0070] selectedTrj: stores the selected best path.

[0071] bestReward: Stores the best reward value currently found. The initial value is negative infinity.

[0072] bestTrj: stores the path associated with the best reward value.

[0073] For each simulation or search result, compare the reward value reward with the currently recorded best reward bestReward.

[0074] If reward is greater than bestReward, update bestReward to reward, and store the corresponding move m and path Trj as new selectedMove and bestTrj.

[0075] If no better reward is found, keep selectedMove and bestTrj unchanged.

[0076] Apply the selected move selectedMove to the current state s and update the state.

[0077] Add selectedMove to selectedTrj and update the path.

[0078] Step 8: In each round of iteration, the maximum reward value of all movements is recorded, and how to balance the fairness and efficiency of rescue services through the path selection of the drone is considered; the system records the maximum reward value of all movements in each round of iteration, and how to balance the fairness and efficiency of rescue services through the path selection of the drone. The key to this step is to comprehensively consider the reward value of each movement and how to balance the fairness and efficiency of the service in path selection.

[0079] Record the maximum reward value of all moves, and comprehensively consider the reward value of each move. The reward value judgment function is as follows:

[0080]

[0081] In the above formula, and Represent the maximum value of task quantity and energy consumption respectively, , , They represent the energy consumption of flight, calculation, and hovering respectively. The maximum reward value is recorded through the reward value function. The system can dynamically adjust the path selection strategy to maximize service efficiency and ensure fairness.

[0082] Step 9: If the maximum reward value is greater than the current optimal reward value, the optimal reward value is updated and the associated path is selected. Emphasis is placed on the path selection of rescue teams that can improve rescue efficiency while taking into account the large amount of tasks; if the maximum reward value is greater than the current optimal reward value, the system updates the optimal reward value and selects the associated path, emphasizing the path selection of rescue teams that can improve rescue efficiency while taking into account the large amount of tasks. The purpose of this step is to ensure that the system selects the optimal path to improve rescue efficiency and optimize resource utilization while ensuring fairness of service.

[0083] Step 10: After the algorithm completes all iterations, it returns the best reward and the best path. This final result will provide the best computing offloading service solution for ground rescue, ensuring the fairness of the service and optimizing the efficiency of resource utilization; After the algorithm completes all iterations, the system returns the best reward and the best path. This final result will provide the best computing offloading service solution for ground rescue, ensuring the fairness of the service and optimizing the efficiency of resource utilization. By returning the best reward and the best path, the system can provide a practical and executable path planning solution, providing effective support for the operation of the UAV-assisted wireless communication system.

[0084] Figure 2 The figure shows a background of a wireless communication scenario in which a line-of-sight (LoS) communication link between an unmanned aerial vehicle (UAV) and a ground rescue team is envisioned. In this scenario, the channel gain follows the free space path loss model. In this configuration, a single UAV is responsible for providing services to multiple ground rescue teams distributed in a specific area. There are N fixed hovering points set above the disaster area, and the UAV can only hover at these predefined locations to perform computational tasks for the rescue teams in the area. In addition, the UAV will fly horizontally at a fixed altitude H. The total service time T is divided into n independent time periods, and the time intervals are denoted by t=0,1,2,...,n-1. In each time interval, the UAV will move from one hovering point to another, giving priority to providing services to the nearest rescue team. At the beginning of each time interval, all rescue teams will send a mission request containing their location information to the UAV so that the UAV can update the user's location information. At the initial time, t=0, the positions of the rescue teams are randomly determined. But at the beginning of the next time interval, their positions will be randomly adjusted. The height H is fixed, and at the same time, there are N fixed hovering points, which are expressed in three-dimensional coordinates.

[0085] Figure 3 The figure shows the average throughput under different numbers of training sets. Monte Carlo Tree Search (MCTS) is used as a benchmark, the number of simulations is used as the horizontal coordinate, and the experimental data is recorded every 100 times. As the number of training sets increases, the average throughput gradually increases, and the maximum average throughput is reached when the level of NMCS is 2. When the number of training sets = 10, the average throughput is 8.34, and when the number of training sets increases to 1000, the average throughput increases to 12.48. When level = 2, the maximum average throughput is achieved.

[0086] Figure 4The figure shows the performance analysis of the Nested Monte Carlo Search (NMCS) algorithm for different numbers of hovering points (denoted by N). Notably, NMCS performs well and is the best performing algorithm when N=20, 30, and 40. It is worth noting that as N increases, the average throughput also increases. This phenomenon is attributed to the larger decision space provided by the increase in the number of hovering points. Therefore, as the number of hovering points increases, the performance of the algorithm also improves because the drone has more action options. It can be clearly seen that the average throughput continues to rise with the increase in the number of hovering points. In addition, the average throughput reaches the highest when level=2, further highlighting the effectiveness of NMCS in navigating complex scenarios.

[0087] Furthermore, when comparing NMCS with the MCTS algorithm, it is clearly seen that NMCS consistently outperforms MCTS in terms of average throughput at different numbers of hovering points. This indicates that NMCS outperforms MCTS in terms of robustness and in scenarios where throughput maximization is critical.

[0088] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0089] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation modes that those skilled in the art can understand.

Claims

1. A path planning method for a UAV-assisted wireless communication system. Features: The method comprises the following steps: Step 1: Initialize the drone state and the nested level of the nested Monte Carlo search. At this point, set the basic state of the drone, including the initial speed and load the ground rescue team's mission; Step 2: The system initializes the best reward value obtained by the drone and the best path selected, and establishes an evaluation function that takes into account the fairness of the service to ensure that priority is given to the rescue team with a large workload; Step 3: In each iteration, check whether the current drone state is in the terminal state, the drone has run out of power, has covered all rescue teams, or has completed all planned flight missions; Step 4: If the current drone state is not the terminal state, iterate each legal movement of the drone state, consider the location of each rescue team, the amount of tasks, and the current state of the drone, and plan a reasonable flight path; Step 5: For each move, calculate the reward value of the new state, which includes not only the cost of reaching the new location, but also the effect of providing computational offloading services at that location; Step 6: If the nesting level is 1, perform a random simulation to evaluate which strategy is most effective in the search at the lower level by simulating various possible flight and service scenarios; Step 7: If the nesting level is greater than 1, recursively call the nested Monte Carlo search to further refine the search and explore the impact of more levels of decision-making, including how to prioritize rescue teams with large workloads; Step 8: In each iteration, record the maximum reward value of all moves and consider how to balance the fairness and efficiency of rescue services through the path selection of the drone; Step 9: If the maximum reward value is greater than the current optimal reward value, update the optimal reward value and select the path associated with it, focusing on those paths that can improve rescue efficiency while taking into account the rescue teams with large task volumes; Step 10: After the algorithm completes all iterations, it returns the best reward and the best path, providing the best computational offloading service solution for ground rescue, ensuring the fairness of the service and optimizing the efficiency of resource utilization.

2. A method for path planning of a UAV-assisted wireless communication system according to claim 1, It is characterized in that In the step 1, the drone state is first initialized, including setting the basic state of the drone, including the initial speed and loading the task of the ground rescue team. At the same time, the nesting level of the nested Monte Carlo search is initialized.

3. A method for path planning of a UAV-assisted wireless communication system according to claim 1, It is characterized in that In step 2, the system initializes the best reward value obtained by the drone and the best path selected. In this process, an evaluation function is introduced. The evaluation function formula is as follows: ， In the above formula, Represents the number of offload tasks, where is a binary variable indicating the service status of the rescue team, i.e., whether it is receiving service, and θ represents a predetermined threshold of the number of tasks. If the rescue team has received service before and the subsequent task volume exceeds θ, the rescue team will receive service again; This function takes into account the fairness of the service and ensures that priority is given to rescue teams with large workloads. Through the establishment of the evaluation function, the system can comprehensively consider the needs of each rescue team in the path planning process and weigh the fairness and efficiency of the service when selecting the best path.

4. A method for path planning of a UAV-assisted wireless communication system according to claim 1, It is characterized in that In step 7, by recursively calling the search algorithm, the system can optimize the path at a deeper level and comprehensively consider more factors to find the optimal flight path.

5. A method for path planning of a UAV-assisted wireless communication system according to claim 1, It is characterized in that In step 8, the maximum reward value among all moves is recorded, and the reward value of each move is comprehensively considered. The reward value judgment function is as follows: ， In the above formula, and Represent the maximum value of task quantity and energy consumption respectively, , , They represent the energy consumption of flight, calculation, and hovering respectively. The maximum reward value is recorded through the reward value function. The system can dynamically adjust the path selection strategy to maximize service efficiency and ensure fairness.

Images