Unmanned aerial vehicle path planning method and system based on reinforcement learning

Abstract

The invention relates to the field of forest fire prevention unmanned aerial vehicle path planning, in particular to an unmanned aerial vehicle path planning method and system based on reinforcement learning. According to the method, all fire high-incidence areas of a forest are obtained and serve as unmanned aerial vehicle passing nodes; a reference planning radius is obtained according to distance distribution between adjacent nodes, and then all unmanned aerial vehicle paths are obtained; according to the distance between every two adjacent nodes in each path, the wind power characteristics and the state of the unmanned aerial vehicle, obtaining wind power propulsion parameters; empty consumption parameters are obtained according to the unmanned aerial vehicle states between the nodes in the paths, and comprehensive evaluation parameters between every two adjacent unmanned aerial vehicle passing nodes in each unmanned aerial vehicle path can be obtained through combination; and in the DQN algorithm, unmanned aerial vehicle path planning is carried out by using the comprehensive evaluation parameters. The most reasonable unmanned aerial vehicle path can be found, so that the electric quantity loss of the unmanned aerial vehicle is reduced, and the forest fire protection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of path planning for forest fire prevention drones, and particularly to a method and system for path planning of drones based on reinforcement learning. Background Art

[0002] Drones reduce energy consumption through bionic path planning (such as imitating bird flight trajectories) in farmland monitoring and power grid inspection, with the coverage efficiency increased by 40%. Combining three-dimensional dynamic path planning enables rapid response in scenarios such as fire monitoring and earthquake rescue, and some systems already support multi-modal data fusion for obstacle avoidance. Drone path planning is a core part of drone technology, directly affecting its safety, efficiency, and application scope. With the large-scale application of drones in fields such as logistics, agriculture, and rescue, the importance of path planning has become even more prominent. The main research focus of the present invention is on how to plan the drone route through reinforcement learning in forest fire protection. Currently, a technical system of "intelligent algorithm + multi-aircraft collaboration + real-time response" has been formed for the path planning of forest fire prevention drones, but there is still a need to break through the complex environment adaptability and computing power bottlenecks. In the future, through autonomous upgrading and multi-platform integration, drones will become the core defense line for forest fire prevention, promoting the transformation from "passive disaster relief" to "active prevention and control".

[0003] However, in actual situations, there are many factors affecting the flight of drones above forests, such as wind resistance, energy loss of drones caused by flight altitude, and the complexity of the forest itself. In the prior art, when using drones for forest fire prevention, reasonable path planning cannot be carried out, resulting in very serious power consumption of the drones, thus reducing the efficiency of forest fire protection by the drones. Summary of the Invention

[0004] In order to solve the technical problem that there are many factors affecting the flight of drones above the forest, and in the existing technology, when using drones for forest fire prevention, reasonable path planning cannot be carried out, resulting in very serious power consumption of the drones, thus reducing the forest fire protection efficiency of the drones, the purpose of the present invention is to provide a drone path planning method and system based on reinforcement learning, and the specific technical solutions adopted are as follows: A drone path planning method based on reinforcement learning, the method includes: obtaining all high-fire-risk areas in the forest, and taking each of the high-fire-risk areas as a drone passing node; calculating and obtaining the reference planning radius of each drone passing node according to the distance distribution between every two drone passing nodes; designing traversal paths for all drone passing nodes according to the reference planning radius to obtain all possible traversal path sets; arbitrarily selecting a drone path as a reference path; obtaining the wind propulsion parameters between every two adjacent drone passing nodes in the reference path according to the distance between every two adjacent drone passing nodes in the reference path, the drone flight speed, the drone flight time, and the wind characteristics; obtaining the empty consumption parameters of the drone between every two adjacent drone passing nodes in the reference path according to the drone flight height, the drone flight time, and the drone power consumption between every two adjacent drone passing nodes in the reference path; obtaining the comprehensive evaluation parameters between every two adjacent drone passing nodes in the reference path according to the wind propulsion parameters, the empty consumption parameters of the drone, the distance distribution, the drone flight time, and the drone power consumption between every two adjacent drone passing nodes in the reference path; in the DQN algorithm, using the comprehensive evaluation parameters for path planning.

[0005] Further, the method for obtaining the reference planning radius includes: obtaining the reference planning radius according to the reference planning radius calculation formula, and the reference planning radius calculation formula is as follows: In the formula, represents the reference planning radius of each drone passing node; represents the number of drone passing nodes; represents the distance between each drone passing node and the th other drone passing node, is a natural number representing the number of drones.

[0006] Further, traverse the path design for all the drone passing nodes according to the reference planning radius to obtain all possible sets of traversed paths, including: selecting any one of the drone passing nodes as the reference node; taking the reference node as the starting node, using the starting node as the center of a circle and the reference planning radius as the radius to construct a reference planning circle of the starting node, selecting each drone passing node within the reference planning circle as the next connection node of the drone, taking any one of the connection nodes as the new starting node, repeating the above steps until there are no more drone passing nodes in the reference planning circle, connecting the center of the reference planning circle at this time to the drone passing node with the closest distance that has not appeared in the reference planning circle as the latest connection node, taking the latest connection node as the new starting node, repeating the above steps until the drone has passed through all the drone passing nodes, obtaining all the drone paths with the reference node as the starting node; traversing all the drone passing nodes to obtain all the drone paths with each drone passing node as the starting node.

[0007] Further, the method for obtaining the wind propulsion parameter includes: obtaining the wind propulsion parameter according to the wind propulsion parameter calculation formula, and the wind propulsion parameter calculation formula is as follows: In the formula, represents the wind propulsion parameter between the th drone passing node and the th drone passing node in the reference path; represents the average speed from the th drone passing node to the th drone passing node under windless conditions; represents the position coordinate of the th drone passing node; represents the position coordinate of the th drone passing node; represents the moment when the drone reaches the th drone passing node; represents the moment when the drone reaches the th drone passing node; represents the wind force of the drone at position , at moment ; represents the wind direction of the drone at position , at moment ; represents the arctangent function; represents the th drone passing node; represents the Euclidean distance between the position coordinate of the th drone passing node and the position coordinate of the th drone passing node.

[0008] Further, the method for obtaining the idle consumption parameter includes: obtaining all preset flight altitude gear types divided according to the vegetation height requirement below when the UAV flies in the forest; obtaining the power consumption and flight time of the UAV in each flight altitude gear type; multiplying the power consumption and flight time of the UAV in each flight altitude gear type between every two adjacent UAV passing nodes, and accumulating and summing to obtain the idle consumption parameter of the UAV between every two adjacent UAV passing nodes.

[0009] Further, the method for obtaining the comprehensive evaluation parameter includes: obtaining the comprehensive evaluation parameter according to the comprehensive evaluation parameter calculation formula, and the comprehensive evaluation parameter calculation formula is as follows: In the formula, represents the comprehensive evaluation parameter between the th UAV passing node and the th UAV passing node in the reference path; represents the wind propulsion parameter between the th UAV passing node and the th UAV passing node in the reference path; [[ID=,20]]represents the average power consumed by the UAV from the th UAV passing node to the th UAV passing node at the same altitude; represents the moment when the UAV arrives at the th UAV passing node; represents the moment when the UAV arrives at the th UAV passing node; represents the idle consumption parameter of the UAV from the th UAV passing node to the th UAV passing node; represents the th position coordinate of the UAV passing node; represents the th position coordinate of the UAV passing node; represents the maximum distance between every two adjacent nodes in the reference path; represents the number of UAV passing nodes; [[ID=,52]]represents the exponential function with the natural constant as the base.

[0010] A UAV path planning system based on reinforcement learning, the system includes a memory, a processor, and a computer program stored in the memory and operable on the processor, and when the processor executes the computer program, the steps of the above-mentioned UAV path planning method based on reinforcement learning are implemented.

[0011] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned method for path planning of an unmanned aerial vehicle based on reinforcement learning are implemented.

[0012] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned method for path planning of an unmanned aerial vehicle based on reinforcement learning are implemented.

[0013] The present invention has the following beneficial effects: In order to focus on monitoring the forest, all high-fire-risk areas in the forest are obtained; in order to facilitate subsequent path planning for the unmanned aerial vehicle, each high-fire-risk area is used as a node through which the unmanned aerial vehicle passes; in order to ensure efficiency, according to the distance distribution between every two of all the nodes through which the unmanned aerial vehicle passes, the reference planning radius of each node through which the unmanned aerial vehicle passes is calculated, and all the paths of the unmanned aerial vehicle are found through the reference planning radius; since the wind direction and wind force in the forest can affect the power consumption of the unmanned aerial vehicle, the wind propulsion parameters between every two adjacent nodes through which the unmanned aerial vehicle passes in the reference path are analyzed; since the tree height in different areas also has a certain impact on the power consumption of the unmanned aerial vehicle, the empty consumption parameters between every two adjacent nodes through which the unmanned aerial vehicle passes in the reference path are analyzed; according to the wind propulsion parameters, the empty consumption parameters, the distance distribution, the flight time of the unmanned aerial vehicle, and the power consumption of the unmanned aerial vehicle between every two adjacent nodes through which the unmanned aerial vehicle passes in the reference path, the comprehensive evaluation parameters between every two adjacent nodes through which the unmanned aerial vehicle passes in the reference path are obtained; in the DQN algorithm, the comprehensive evaluation parameters are used for path planning. The present invention can use the DQN algorithm to find the most reasonable path of the unmanned aerial vehicle, thereby reducing the power loss of the unmanned aerial vehicle and improving the efficiency of forest fire protection. Description of the Drawings

[0014] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0015] Figure 1 It is a flowchart of a method for path planning of an unmanned aerial vehicle based on reinforcement learning provided by an embodiment of the present invention; Figure 2 It is a block diagram of a system for path planning of an unmanned aerial vehicle based on reinforcement learning provided by an embodiment of the present invention. Detailed Embodiments

[0016] In order to further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following specifically describes, with reference to the accompanying drawings and preferred embodiments, a method for unmanned aerial vehicle path planning based on reinforcement learning according to the present invention, including its specific implementation manner, structure, features and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0017] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present invention belongs.

[0018] The following specifically describes the specific solution of a method and system for unmanned aerial vehicle path planning based on reinforcement learning provided by the present invention with reference to the accompanying drawings.

[0019] Please refer to Figure 1 , which shows a method for unmanned aerial vehicle path planning based on reinforcement learning provided by an embodiment of the present invention. The method includes: Step S1: Obtain all high-fire-occurrence areas in the forest, and use each high-fire-occurrence area as a node for the unmanned aerial vehicle to pass through.

[0020] The embodiments of the present invention are mainly applied to the scenario of forest fire prevention by unmanned aerial vehicles. Since the key to forest fire prevention and control lies in being able to detect the fire immediately when it just occurs and extinguish it before a forest fire is formed, so as to avoid a large consumption of manpower and material resources and protect more forest resources from being damaged. However, detecting all forest areas every day will consume a large amount of manpower and material resources, and will also waste time and energy by detecting the wet areas in the forest that are not easily ignited. Therefore, when conducting forest fire prevention and control detection, it is necessary to first identify the key detection areas. In the prior art, it can be found through a high-definition camera carried by an unmanned aerial vehicle that hidden fire sources or smoke are often found in certain fixed areas in the forest. For example, in the forest, areas covered by highly flammable vegetation, such as bushes and coniferous forests, are prone to catching fire due to their plant characteristics. Narrow valleys are prone to forming "wind channels" that fuel the fire; the leeward slopes are prone to accumulating dead branches and forming high-load combustibles. Areas with frequent human activities, such as farmlands, camping sites, and roadsides (such as discarded cigarette butts and campfire embers) at the edge of the forest, are areas with a high incidence of human-caused fires. Therefore, in the embodiments of the present invention, the areas covered by highly flammable vegetation, narrow valleys, leeward slopes, farmlands, camping sites, and roadsides at the edge of the forest are used as high-fire-occurrence areas. It should be noted that the high-fire-occurrence areas are not fixed and are not limited here.

[0021] In order to facilitate subsequent path planning for the drone, each high-fire-occurrence area is regarded as a drone passing node, and the best flight path is found through the flight state of the drone and the node distribution between different drone passing nodes. The specific steps are as follows.

[0022] Step S2: Calculate the reference planning radius of each drone passing node according to the distance distribution between every two drone passing nodes; design paths for all drone passing nodes according to the reference planning radius to obtain all drone paths; arbitrarily select one drone path as the reference path; obtain the wind propulsion parameter between every two adjacent drone passing nodes in the reference path according to the distance, drone flight speed, drone flight time, and wind characteristics between every two adjacent drone passing nodes in the reference path; obtain the empty consumption parameter of the drone between every two adjacent drone passing nodes in the reference path according to the drone flight height, drone flight time, and drone power consumption between every two adjacent drone passing nodes in the reference path; obtain the comprehensive evaluation parameter between every two adjacent drone passing nodes in the reference path according to the wind propulsion parameter, empty consumption parameter, distance distribution, drone flight time, and drone power consumption between every two adjacent drone passing nodes in the reference path.

[0023] In actual situations, regarding the drone as an agent requires screening out the most suitable path after reinforcement learning in a forest environment. Before screening out the most suitable path, there are multiple path options for the drone to traverse all drone passing nodes. However, from the perspective of ensuring efficiency, after the drone detects the previous high-fire-occurrence area, it is necessary to select a high-fire-occurrence area relatively close to this high-fire-occurrence area from other high-fire-occurrence areas as the next detection area for the drone. Therefore, in the embodiments of the present invention, the reference planning radius of each drone passing node is calculated according to the distance distribution between every two drone passing nodes, and all drone paths are found through the reference planning radius.

[0024] Preferably, in an embodiment of the present invention, the method for obtaining the reference planning radius includes: obtaining the reference planning radius according to the reference planning radius calculation formula, and the reference planning radius calculation formula is as follows: In the formula, represents the reference planning radius of each drone passing node; represents the number of drone passing nodes; represents the distance between each drone passing node and the th other drone passing node, is a natural number representing the number of drones.

[0025] In the reference planning radius calculation formula, calculating the average distance between a certain drone passing node and other drone passing nodes can reflect the approximate distance between the drone passing node and other drone passing nodes. If the distance between another drone passing node and the drone passing node is greater than the average distance, it is considered that the distance between the two nodes is relatively far and is not used as the next detection node for the drone. Since there may be drone passing nodes that are far from all other drone passing nodes, resulting in a large error in the reference planning radius. To reduce the error, the average value of the average distances between each drone passing node and other drone passing nodes is calculated as the final reference planning radius.

[0026] Preferably, in an embodiment of the present invention, path design is performed on all drone passing nodes according to the reference planning radius to obtain all drone paths, including: arbitrarily selecting a drone passing node as a reference node; using the reference node as the starting node, with the starting node as the center and the reference planning radius as the radius, constructing a reference planning circle of the starting node, and selecting each drone passing node within the reference planning circle as the next connection node of the drone. Here, each drone passing node within the reference planning circle can be used as the next connection node; using any one of the connection nodes as the new starting node, repeating the above steps until there are no more drone passing nodes in the reference planning circle. At this time, connect the center of the reference planning circle to the drone passing node with the closest distance that does not appear in the reference planning circle as the latest connection node, use the latest connection node as the new starting node, and repeat the above steps until the drone passes through all drone passing nodes to obtain all drone paths with the reference node as the starting node; traverse all drone passing nodes to obtain all drone paths with each drone passing node as the starting node.

[0027] When a drone is performing forest fire prevention and detection operations, since all flammable areas need to be detected, if the detection is carried out directly without route planning, it is very easy for the drone to run out of power during the detection process due to unreasonable route arrangements, or for the fire to not be detected in a timely manner due to excessive time consumption. Therefore, it is necessary to find the best path among all drone paths. The basis for finding the best path lies in how to detect all the high-fire-risk areas in the forest with as low energy consumption as possible. Among the factors causing the drone's power consumption, in addition to normal flight to perform tasks, the wind direction in the forest is one of the most important influencing factors. In the embodiment of the present invention, a three-dimensional ultrasonic anemometer is equipped for the drone, so that the drone can collect the magnitude and direction data of the wind force it receives in real time while flying. If the wind direction in the forest is relatively consistent with the flight direction of the drone, the wind force will also drive the flight of the drone, thus saving more power. And if the flight direction of the drone is opposite to the wind direction, the drone will need to consume more power to complete the detection task. Therefore, in the embodiment of the present invention, first, an arbitrary drone path is selected as the reference path, and the wind force propulsion parameters between every two adjacent drone passing nodes in the reference path are analyzed.

[0028] Preferably, in an embodiment of the present invention, the method for obtaining the wind force propulsion parameters includes: obtaining the wind force propulsion parameters according to the wind force propulsion parameter calculation formula, and the wind force propulsion parameter calculation formula is as follows: In the formula, represents the wind force propulsion parameter between the th drone passing node and the th drone passing node in the reference path; represents the average speed of flying from the th drone passing node to the th drone passing node in the absence of wind; represents the position coordinate of the th drone passing node; represents the position coordinate of the th drone passing node; represents the moment when the drone reaches the th drone passing node; represents the moment when the drone reaches the th drone passing node; represents the wind force of the drone at the position , at the moment ; represents the wind direction of the drone at the position , at the moment ; represents the arctangent function; represents the The Euclidean distance between the position coordinates of the th node passed by the drone and the position coordinates of the

[0029] In the calculation formula of the wind propulsion parameter, when the drone is flying between two nodes passed by the drone, the angle between the wind direction and the drone will change with the changes of the time scale and the space scale. Similarly, the magnitude of the wind force will also change with the changes of the two scales. Therefore, the of the wind force, and the angle between the wind direction and the drone are both functions of the space scale and the time scale . Therefore, obtaining , and if the angle between the wind direction and the flight direction of the drone is less than 90°, it means that the wind force has a pushing effect on the drone, and the greater the wind force, the stronger the pushing effect. On the contrary, if the angle between the wind direction and the flight direction of the drone is greater than 90°, it means that the wind force has an obstructive effect on the flight of the drone, and the greater the wind force, the stronger the obstructive effect. Therefore, is larger, and at this time the wind force is greater, indicating that the propulsion effect of the wind force is stronger. For all the wind forces and wind directions between the th node passed by the drone and the th node passed by the drone, a double integral is performed to obtain to represent the propulsion effect of the wind force on the drone between the th node passed by the drone and the th node passed by the drone; in the state of no wind, from the th node passed by the drone to the th node passed by the drone, the average speed is faster, the Euclidean distance between the two nodes is smaller, and the th node passed by the drone and the th node passed by the drone, the flight time is smaller, indicating that the flight efficiency of the drone in the state of no wind is higher. And taking the propulsion effect of the wind force on the drone as the weight, the greater the weight, the greater the wind propulsion parameter. If the weight is less than 0, it means that the wind force has an obstructive effect on the drone. Since in the subsequent process, in order to keep the result positive, the result obtained by the integral is put into the function to ensure that the subsequent result is positive. Then the greater the result at this time, the stronger the beneficial effect on the drone.

[0030] It should be noted that when the drone is between the th node passed by the drone and the The average speed between the nodes passed by a drone, the corresponding time of the nodes passed by the th drone and the nodes passed by the

[0031] th drone can be obtained through the drone's own system and will not be elaborated here.

[0032] Preferably, in an embodiment of the present invention, the method for obtaining the empty consumption parameter includes: obtaining all preset flight altitude gear types divided according to the vegetation height requirements below when the drone flies in the forest. The flight altitude gear type is the flight altitude required when the drone encounters different tree types. The specific content is a well-known technical means for those skilled in the art and will not be elaborated here.

[0033] The power consumption of the drone and the flight time at each flight altitude gear type of the drone can be directly obtained through the background system of the drone and will not be elaborated here.

[0034] First, obtain the flight altitude gear type between every two adjacent nodes passed by the drone. Multiply the power consumption of the drone at each flight altitude gear type between every two adjacent nodes passed by the drone by the flight time and accumulate the sum to obtain the empty consumption parameter between every two adjacent nodes passed by the drone.

[0035] According to the wind propulsion parameter, the empty consumption parameter, the distance distribution, the flight time of the drone, and the power consumption of the drone between every two adjacent nodes passed by the drone in the reference path, obtain the comprehensive evaluation parameter of the reference path.

[0036] Preferably, in an embodiment of the present invention, the method for obtaining the comprehensive evaluation parameter includes: obtaining the comprehensive evaluation parameter according to the comprehensive evaluation parameter calculation formula. The comprehensive evaluation parameter calculation formula is as follows: In the formula, represents the comprehensive evaluation parameter between the th node passed by the drone and the th node passed by the drone in the reference path; represents the th node passed by the drone and the The wind propulsion parameters between the passing nodes of the UAV; Indicates that at the same altitude, from the th passing node of the UAV to the th passing node of the UAV, the average power consumed; Indicates the moment when the UAV reaches the th passing node of the UAV; Indicates the moment when the UAV reaches the th passing node of the UAV; Indicates that from the th passing node of the UAV to the th passing node of the UAV, the idle consumption parameter; Indicates the th position coordinate of the passing node of the UAV; Indicates the th position coordinate of the passing node of the UAV; Indicates the maximum distance between every two adjacent nodes in the reference path; Indicates the number of passing nodes of the UAV; Indicates the exponential function with the natural constant as the base.

[0037] In the calculation formula of the comprehensive evaluation parameter, since the power consumption of the UAV under different wind forces is different, the product of the average power consumption of the UAV and the flight time between the th passing node of the UAV and the th passing node of the UAV is used as the power consumption of the UAV between the th passing node of the UAV and the th passing node of the UAV; and the larger the ratio of the wind propulsion parameter between the th passing node of the UAV and the th passing node of the UAV to the power consumption of the UAV, the more favorable the wind is for the flight of the UAV between the th passing node of the UAV and the th passing node of the UAV; since the smaller the idle consumption parameter, the smaller the power consumption of the UAV between two nodes, and the smaller the Euclidean distance between two nodes, the more reasonable the path planning between two nodes, and using to control the value range.

[0038] Step S3: In the DQN algorithm, use the comprehensive evaluation parameter for path planning.

[0039] In an embodiment of the present invention, in the DQN algorithm, path planning is performed according to comprehensive evaluation parameters, including: when using the DQN algorithm to perform path planning for a drone, input the state data of the drone at different spatial positions, such as the position coordinate information of the drone, the surrounding environment information, the wind force information, the energy consumption information, etc. It should be noted that the above information can be obtained from the system parameters of the drone and will not be elaborated here. When the drone is at a certain drone passing node, input the position coordinate information, the surrounding environment information of the drone passing node, and the comprehensive evaluation parameter between the drone passing node and the next adjacent drone passing node in all drone paths into the DQN algorithm. Generate the corresponding Q value through these parameters, and use the next adjacent drone passing node with the largest Q value as the drone passing node that the drone needs to reach next. By analogy, until all the drone passing nodes passed by the drone are selected, thus completing the path planning of the drone. It should be noted that the DQN algorithm is a well-known technical means to those skilled in the art and will not be elaborated here.

[0040] In summary, obtain all the high-fire-occurrence areas in the forest, and use each high-fire-occurrence area as a drone passing node; calculate the reference planning radius of each drone passing node according to the distance distribution between every two drone passing nodes; perform path design on all drone passing nodes according to the reference planning radius to obtain all drone paths; arbitrarily select one drone path as the reference path; obtain the wind force propulsion parameter between every two adjacent drone passing nodes in the reference path according to the distance between every two adjacent drone passing nodes in the reference path, the drone flight speed, the drone flight time, and the wind force characteristics; obtain the empty consumption parameter of the drone between every two adjacent drone passing nodes in the reference path according to the drone flight height, the drone flight time, and the drone consumption power between every two adjacent drone passing nodes in the reference path; obtain the comprehensive evaluation parameter between every two adjacent drone passing nodes in the reference path according to the wind force propulsion parameter, the empty consumption parameter of the drone, the distance distribution, the drone flight time, and the drone consumption power between every two adjacent drone passing nodes in the reference path; in the DQN algorithm, use the comprehensive evaluation parameter for path planning.

[0041] An embodiment of the present invention provides a drone path planning system based on reinforcement learning. The system includes a memory, a processor, and a computer program, where the memory is used to store the corresponding computer program, the processor is used to run the corresponding computer program, and when the computer program runs in the processor, it can implement the method described in steps S1 - S3, specifically as Figure 2 shown, including: a node acquisition module 101, configured to obtain all the high-fire-occurrence areas in the forest, and use each high-fire-occurrence area as a drone passing node.

[0042] The UAV path analysis module 102 is configured to calculate and obtain the reference planning radius of each UAV passing node according to the distance distribution between every two UAV passing nodes; perform path design on all UAV passing nodes according to the reference planning radius to obtain all UAV paths; randomly select one UAV path as the reference path; obtain the wind propulsion parameters between every two adjacent UAV passing nodes in the reference path according to the distance, UAV flight speed, UAV flight time, and wind characteristics between every two adjacent UAV passing nodes in the reference path; obtain the empty consumption parameters of the UAV between every two adjacent UAV passing nodes in the reference path according to the UAV flight altitude, UAV flight time, and UAV power consumption between every two adjacent UAV passing nodes in the reference path; and obtain the comprehensive evaluation parameters between every two adjacent UAV passing nodes in the reference path according to the wind propulsion parameters, empty consumption parameters, distance distribution, UAV flight time, and UAV power consumption between every two adjacent UAV passing nodes in the reference path.

[0043] The path planning module 103 is configured to perform path planning using the comprehensive evaluation parameters in the DQN algorithm.

[0044] The third object of the embodiments of the present invention is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the methods described in steps S1-S3 are implemented.

[0045] The fourth object of the embodiments of the present invention is to provide a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the methods described in steps S1-S3 are implemented.

[0046] It should be noted that the above sequence of the embodiments of the present invention is only for description and does not represent the superiority or inferiority of the embodiments. The processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0047] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments.

Claims

1. A method for path planning of unmanned aerial vehicles based on reinforcement learning, characterized in that, The method includes: obtaining all high-fire-incidence areas in the forest, and taking each of the high-fire-incidence areas as a drone passing node; calculating a reference planning radius for each drone passing node according to the distance distribution between every two drone passing nodes; designing traversal paths for all drone passing nodes according to the reference planning radius to obtain all possible drone path sets; randomly selecting one drone path as a reference path; obtaining a wind propulsion parameter between every two adjacent drone passing nodes in the reference path according to the distance between every two adjacent drone passing nodes in the reference path, the drone flight speed, the drone flight time, and the wind characteristics; obtaining an empty consumption parameter of the drone between every two adjacent drone passing nodes in the reference path according to the drone flight height, the drone flight time, and the drone power consumption between every two adjacent drone passing nodes in the reference path; obtaining a comprehensive evaluation parameter between every two adjacent drone passing nodes in the reference path according to the wind propulsion parameter, the empty consumption parameter of the drone, the distance distribution, the drone flight time, and the drone power consumption between every two adjacent drone passing nodes in the reference path; and performing path planning using the comprehensive evaluation parameter in the DQN algorithm.

2. The method for path planning of an unmanned aerial vehicle based on reinforcement learning according to claim 1, wherein The method for obtaining the reference planning radius includes: obtaining the reference planning radius according to the reference planning radius calculation formula, and the reference planning radius calculation formula is as follows: In the formula, represents the reference planning radius of each UAV passing through a node; represents the number of UAV passing through nodes, which is greater than 1; represents the distance between each UAV passing through a node and the th other UAV passing through node, where is a natural number representing the number of UAVs.

3. The method for path planning of an unmanned aerial vehicle based on reinforcement learning according to claim 1, wherein The method for designing traversal paths for all drone passing nodes according to the reference planning radius to obtain all possible traversal path sets includes: randomly selecting one drone passing node as a reference node; taking the reference node as a starting node, using the starting node as the center of a circle and the reference planning radius as the radius to construct a reference planning circle of the starting node, selecting each drone passing node within the reference planning circle as the next connection node of the drone, taking any one of the connection nodes as a new starting node, and repeating the above steps until there are no more drone passing nodes in the reference planning circle; connecting the center of the current reference planning circle to the drone passing node that has not appeared in the generated path and is the closest to the starting node as the latest connection node, taking the latest connection node as a new starting node, and repeating the above steps until the drone passes through all drone passing nodes to obtain all drone paths with the reference node as the starting node; and traversing all drone passing nodes to obtain all drone paths with each drone passing node as the starting node.

4. A method for path planning of an unmanned aerial vehicle based on reinforcement learning according to claim 1, characterized in that, The method for obtaining the wind propulsion parameters includes: obtaining the wind propulsion parameters according to the wind propulsion parameter calculation formula, and the wind propulsion parameter calculation formula is as follows: In the formula, represents the wind propulsion parameter between the th and the th drone passing nodes on the reference path; represents the average speed of flying from the th drone passing node to the th drone passing node under windless conditions; represents the position coordinate of the th drone passing node; represents the position coordinate of the th drone passing node; represents the moment when the drone reaches the th drone passing node; represents the moment when the drone reaches the th drone passing node; represents the wind force of the drone at position , and at moment ; represents the wind direction of the drone at position , and at moment ; represents the arctangent function; represents the th drone passing node's position coordinate and the th drone passing node's position coordinate's Euclidean distance.

5. A method for path planning of an unmanned aerial vehicle based on reinforcement learning according to claim 1, characterized in that, The method for obtaining the empty consumption parameter includes: obtaining all preset flight height gear types divided according to the vegetation height requirements below when the drone flies in the forest; obtaining the drone power consumption and flight time of the drone in each flight height gear type; multiplying the drone power consumption of each flight height gear type between every two adjacent drone passing nodes by the flight time and accumulating the sum to obtain the empty consumption parameter of the drone between every two adjacent drone passing nodes.

6. A method for path planning of an unmanned aerial vehicle based on reinforcement learning according to claim 1, wherein, The method for obtaining the comprehensive evaluation parameter includes: obtaining the comprehensive evaluation parameter according to the comprehensive evaluation parameter calculation formula, and the comprehensive evaluation parameter calculation formula is as follows: In the formula, represents the comprehensive evaluation parameter between the th and the th drone passing nodes in the reference path; represents the wind propulsion parameter between the th and the th drone passing nodes in the reference path; represents the average power consumed by the drone from the th drone passing node to the th drone passing node at the same altitude; represents the moment when the drone reaches the th drone passing node; represents the moment when the drone reaches the th drone passing node; represents the idle consumption parameter of the drone from the th drone passing node to the th drone passing node; represents the position coordinate of the th drone passing node; represents the position coordinate of the th drone passing node; represents the maximum distance between every two adjacent nodes in the reference path; represents the number of drone passing nodes; represents the exponential function with the natural constant as the base.

7. A drone path planning system based on reinforcement learning, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method for drone path planning based on reinforcement learning according to any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of a method for path planning of an unmanned aerial vehicle based on reinforcement learning as described in any one of claims 1 to 6.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that When the processor executes the computer program, it implements the steps of a method for path planning of an unmanned aerial vehicle based on reinforcement learning as described in any one of claims 1 to 6.

Images