A UAV path planning method and system based on reinforcement learning

Abstract

The present invention relates to the field of forest fire prevention drone path planning, and specifically to a drone path planning method and system based on reinforcement learning. The present invention obtains all high-incidence forest fire areas and uses them as drone path nodes; obtains a reference planning radius based on the distance distribution between adjacent nodes, and then obtains all drone paths; obtains wind propulsion parameters based on the distance between each two adjacent nodes in each path, wind characteristics, and the drone's own state; obtains air consumption parameters based on the drone state between the nodes in the path, and combines them to obtain comprehensive evaluation parameters between two adjacent drone path nodes in each drone path; and uses the comprehensive evaluation parameters to perform drone path planning in the DQN algorithm. The present invention can find the most reasonable drone path, thereby reducing drone power loss and improving forest fire prevention efficiency.

Description

Technical Field

[0001] The present invention relates to the field of forest fire prevention UAV path planning, and in particular to a UAV path planning method and system based on reinforcement learning. Background Art

[0002] Drones used in farmland monitoring and power grid inspections use biomimetic path planning (e.g., mimicking bird flight paths) to reduce energy consumption and increase coverage efficiency by 40%. Combined with three-dimensional dynamic path planning, they enable rapid response in scenarios such as fire monitoring and earthquake rescue. Some systems already support multimodal data fusion and obstacle avoidance. Drone path planning is a core component of drone technology, directly impacting its safety, efficiency, and scope of application. With the large-scale application of drones in logistics, agriculture, and rescue operations, the importance of path planning has become increasingly prominent. This paper focuses on using reinforcement learning to plan drone routes for forest fire prevention. Currently, drone path planning for forest fire prevention has developed a technical system combining "intelligent algorithms, multi-machine collaboration, and real-time response," but challenges remain in adapting to complex environments and addressing computing power bottlenecks. In the future, through autonomous upgrades and multi-platform integration, drones will become a core line of defense in forest fire prevention, driving the transition from "passive disaster relief" to "active prevention and control."

[0003] However, in reality, numerous factors affect drone flight above forests, including wind resistance, energy loss due to altitude, and the inherent complexity of the forest. Existing technologies for using drones for forest fire prevention lack proper path planning, resulting in significant battery loss and reduced effectiveness. Summary of the Invention

[0004] In order to solve the problem that there are many factors that affect the flight of drones above the forest, and in the existing technology, when drones are used for forest fire prevention, reasonable path planning cannot be carried out, which makes the power consumption of drones very serious, thereby reducing the forest fire protection efficiency of drones, the purpose of the present invention is to provide a drone path planning method and system based on reinforcement learning, and the technical scheme adopted is as follows: a drone path planning method based on reinforcement learning, the method comprises: obtaining all fire-prone areas in the forest, and taking each of the fire-prone areas as a drone passing node; calculating the reference planning radius of each drone passing node according to the distance distribution between all drone passing nodes; traversal path design for all drone passing nodes according to the reference planning radius to obtain all possible traversal path sets; optional A UAV path is used as a reference path; based on the distance between each two adjacent UAV path nodes in the reference path, the UAV flight speed, the UAV flight time and the wind characteristics, the wind propulsion parameters between each two adjacent UAV path nodes in the reference path are obtained; based on the UAV flight altitude, the UAV flight time and the UAV power consumption between each two adjacent UAV path nodes in the reference path, the air consumption parameters of the UAV are obtained; based on the wind propulsion parameters between each two adjacent UAV path nodes in the reference path, the UAV air consumption parameters, the distance distribution, the UAV flight time and the UAV power consumption, the comprehensive evaluation parameters between each two adjacent UAV path nodes in the reference path are obtained; in the DQN algorithm, the comprehensive evaluation parameters are used for path planning.

[0005] Furthermore, the method for obtaining the reference planning radius includes: obtaining the reference planning radius according to a reference planning radius calculation formula, and the reference planning radius calculation formula is as follows: Where, Indicates the reference planning radius of each node that the UAV passes through; Indicates the number of nodes that the drone passes through; Indicates that each drone passes through the node The distance between the nodes passed by other drones, is a natural number representing the number of drones.

[0006] Furthermore, a traversal path design is performed on all the drone passing nodes according to the reference planning radius to obtain a set of all possible traversal paths, including: selecting any drone passing node as a reference node; taking the reference node as the starting node, the starting node as the center of the circle, and the reference planning radius as the radius to construct a reference planning circle of the starting node, selecting each drone passing node in the reference planning circle as the next connection node of the drone, and taking any connection node as the new starting node, repeating the above steps until there are no more drone passing nodes in the reference planning circle, connecting the center of the reference planning circle at this time with the drone passing node closest to the reference planning circle that does not appear in the reference planning circle as the latest connection node, and taking the latest connection node as the new starting node, repeating the above steps until the drone passes through all drone passing nodes to obtain all drone paths with the reference node as the starting node; traversing all drone passing nodes to obtain all drone paths with each drone passing node as the starting node.

[0007] Furthermore, the method for obtaining the wind propulsion parameter includes: obtaining the wind propulsion parameter according to a wind propulsion parameter calculation formula, and the wind propulsion parameter calculation formula is as follows: Where, Indicates the first The drone passes through the node Wind propulsion parameters between nodes that the UAV passes through; In the absence of wind, The drone flew through the node to the The average speed of each drone passing through the node; Indicates the The location coordinates of the nodes that the drone passes through; Indicates the The location coordinates of the nodes that the drone passes through; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates the drone is at position ,time wind force; Indicates the drone is at position ,time wind direction; represents the inverse tangent function; Indicates the The position coordinates of the nodes that the UAV passes through are The Euclidean distance between the position coordinates of the nodes passed by the drone.

[0008] Furthermore, the method for obtaining the idle consumption parameters includes: obtaining all preset flight altitude gear types divided according to the height requirements of the vegetation below when the drone is flying in the forest; obtaining the drone power consumption and flight time of the drone in each flight altitude gear type; multiplying the drone power consumption and flight time of each flight altitude gear type between each two adjacent drone passing nodes, and accumulating the sum to obtain the idle consumption parameters of the drone between each two adjacent drone passing nodes.

[0009] Furthermore, the method for obtaining the comprehensive evaluation parameter includes: obtaining the comprehensive evaluation parameter according to a comprehensive evaluation parameter calculation formula, and the comprehensive evaluation parameter calculation formula is as follows: Where, Indicates the first The drone passes through the node Comprehensive evaluation parameters between nodes that UAVs pass through; Indicates the first The drone passes through the node Wind propulsion parameters between nodes that the UAV passes through; Indicates that the drone is at the same altitude. The drone passes through the node to the The average power consumed by each drone passing through a node; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates that the drone is The drone passes through the node to the The idle consumption parameters of the nodes that the UAV passes through; Indicates the The location coordinates of the nodes that the drone passes through; Indicates the The location coordinates of the nodes that the drone passes through; Indicates the maximum distance between every two adjacent nodes in the reference path; Indicates the number of nodes that the drone passes through; Represents an exponential function with a natural constant as its base.

[0010] A reinforcement learning-based drone path planning system, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the reinforcement learning-based drone path planning method described above when executing the computer program.

[0011] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned reinforcement learning-based drone path planning method.

[0012] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned reinforcement learning-based drone path planning method are implemented.

[0013] The present invention has the following beneficial effects: in order to carry out key monitoring of forests, the present invention obtains all high-incidence areas in the forest; in order to facilitate subsequent path planning for drones, each high-incidence area is used as a drone pass-through node; in order to ensure efficiency, the reference planning radius of each drone pass-through node is calculated based on the distance distribution between all drone pass-through nodes, and all drone paths are found based on the reference planning radius; since the wind direction and wind force in the forest can affect the power consumption of drones, the wind propulsion parameters between each two adjacent drone pass-through nodes in the reference path are analyzed; since the height of trees in different areas also has a certain impact on the power consumption of drones, the air consumption parameters between each two adjacent drone pass-through nodes in the reference path are analyzed; based on the wind propulsion parameters, the air consumption parameters of the drones, the distance distribution, the drone flight time, and the drone power consumption between each two adjacent drone pass-through nodes in the reference path, a comprehensive evaluation parameter between each two adjacent drone pass-through nodes in the reference path is obtained; and in the DQN algorithm, the comprehensive evaluation parameter is used for path planning. The present invention can use the DQN algorithm to find the most reasonable drone path, thereby reducing drone power consumption and improving forest fire prevention efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 A flow chart of a UAV path planning method based on reinforcement learning provided by one embodiment of the present invention; Figure 2 A block diagram of a reinforcement learning drone path planning system provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0016] To further illustrate the technical means and effectiveness of the present invention in achieving its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a reinforcement learning-based drone path planning method proposed by the present invention. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0017] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0018] The following describes in detail a specific solution of a UAV path planning method and system based on reinforcement learning provided by the present invention with reference to the accompanying drawings.

[0019] See also Figure 1 , which shows a UAV path planning method based on reinforcement learning provided by an embodiment of the present invention, the method comprising: step S1: obtaining all high-incidence areas of fire in the forest, and taking each high-incidence area of ​​fire as a node passed by the UAV.

[0020] The embodiments of the present invention are primarily applicable to forest fire prevention using drones. Forest fire prevention and control focuses on detecting fires as soon as they occur and extinguishing them before they become a forest fire. This avoids significant human and material resources and protects more forest resources. However, daily inspections of all forest areas consume significant manpower and resources, and also waste time and energy by ineffectively inspecting moist areas of the forest that are less susceptible to fire. Therefore, when conducting forest fire prevention and control inspections, it is necessary to first identify key inspection areas. Existing drones equipped with high-definition cameras can often reveal hidden fire sources or smoke in certain fixed areas within the forest. For example, areas covered by highly flammable vegetation, such as shrubs and coniferous forests, are prone to ignition due to their vegetative properties. Narrow valleys easily form "wind channels," fueling fires; leeward slopes are prone to the accumulation of dead branches, resulting in high levels of combustible material. Areas with frequent human activity, such as farmland at the edge of the forest, campsites, and roadside areas (e.g., discarded cigarette butts and campfire embers), are high-risk areas for human-caused fires. Therefore, in the embodiment of the present invention, areas covered by highly flammable vegetation, narrow valleys, leeward slopes, farmland at the edge of the forest, campsites, and road surroundings are regarded as high-incidence areas for fire. It should be noted that high-incidence areas for fire are not fixed and are not limited here.

[0021] To facilitate subsequent drone path planning, each fire-prone area is treated as a drone transit node. The optimal flight path is found by analyzing the flight status and node distribution between different drone transit nodes. The specific steps are as follows.

[0022] Step S2: According to the distance distribution between all the drone path nodes, the reference planning radius of each drone path node is calculated; according to the reference planning radius, the path design is performed for all the drone path nodes to obtain all the drone paths; any drone path is selected as the reference path; according to the distance between each two adjacent drone path nodes in the reference path, the drone flight speed, the drone flight time and the wind characteristics, the wind propulsion parameters between each two adjacent drone path nodes in the reference path are obtained; according to the drone flight altitude, the drone flight time and the drone power consumption between each two adjacent drone path nodes in the reference path, the idle consumption parameters of the drone between each two adjacent drone path nodes in the reference path are obtained; according to the wind propulsion parameters, the idle consumption parameters of the drone, the distance distribution, the drone flight time and the drone power consumption between each two adjacent drone path nodes in the reference path, the comprehensive evaluation parameters between each two adjacent drone path nodes in the reference path are obtained.

[0023] In actual situations, using drones as intelligent agents requires screening out the most suitable path after reinforcement learning in a forest environment. Before screening out the most suitable path, the drone has multiple path options for traversing all the nodes it passes through. However, from the perspective of ensuring efficiency, after the drone has detected the previous high-incidence area, it is necessary to select a high-incidence area that is relatively close to the previous high-incidence area among other high-incidence areas as the next detection area of ​​the drone. Therefore, in an embodiment of the present invention, based on the distance distribution between all the nodes passed by the drone, the reference planning radius of each drone passing node is calculated, and all drone paths are found through the reference planning radius.

[0024] Preferably, in one embodiment of the present invention, the method for obtaining the reference planning radius includes: obtaining the reference planning radius according to a reference planning radius calculation formula, the reference planning radius calculation formula being as follows: Where, Indicates the reference planning radius of each node that the UAV passes through; Indicates the number of nodes that the drone passes through; Indicates that each drone passes through the node The distance between the nodes passed by other drones, is a natural number representing the number of drones.

[0025] In the reference planning radius calculation formula, the average distance between a certain drone passing node and other drone passing nodes can be calculated to reflect the approximate distance between the drone passing node and other drone passing nodes. If the distance between the drone passing node and other drone passing nodes is greater than the average distance, it is considered that the distance between the two nodes is relatively far and will not be used as the next detection node of the drone. Since there may be a drone passing node that is far away from all other drone passing nodes, As a reference, the planning radius has a large error. In order to reduce the error, the average distance between each drone passing through the node and the average distance between other drones passing through the node is calculated. , as the final reference planning radius.

[0026] Preferably, in one embodiment of the present invention, path design is performed on all drone path nodes according to the reference planning radius to obtain all drone paths, including: selecting any drone path node as a reference node; taking the reference node as the starting node, the starting node as the center of the circle, and the reference planning radius as the radius to construct a reference planning circle of the starting node, and selecting each drone path node within the reference planning circle as the next connection node of the drone, where each drone path node within the reference planning circle can be used as the next connection node; taking any connection node as the new starting node, repeating the above steps until there are no more drone path nodes in the reference planning circle, connecting the center of the reference planning circle at this time with the drone path node closest to the reference planning circle that does not appear in the reference planning circle as the latest connection node, taking the latest connection node as the new starting node, and repeating the above steps until the drone passes through all drone path nodes, thereby obtaining all drone paths with the reference node as the starting node; traversing all drone path nodes to obtain all drone paths with each drone path node as the starting node.

[0027] When drones perform forest fire prevention and detection operations, they need to inspect all flammable areas. If they proceed directly without route planning, improper route planning can easily lead to the drone running out of power during the inspection process or delaying detection of fires. Therefore, it is necessary to find the optimal path among all available drone paths. The key to finding the optimal path is to ensure that all fire-prone areas in the forest are inspected with the lowest possible energy consumption. Besides normal flight missions, forest wind direction is one of the most significant factors affecting drone power consumption. In one embodiment of the present invention, a drone is equipped with a three-dimensional ultrasonic anemometer, enabling it to collect real-time data on the magnitude and direction of wind forces acting on it during flight. If the wind direction in the forest is consistent with the drone's flight direction, the wind will propel the drone's flight, saving further energy. However, if the drone's flight direction is opposite to the wind direction, the drone will consume more energy to complete the inspection mission. Therefore, in this embodiment of the present invention, a drone path is first selected as a reference path, and the wind propulsion parameters between each two adjacent nodes along the reference path are analyzed.

[0028] Preferably, in one embodiment of the present invention, the method for obtaining wind propulsion parameters includes: obtaining wind propulsion parameters according to a wind propulsion parameter calculation formula, and the wind propulsion parameter calculation formula is as follows: Where, Indicates the first The drone passes through the node Wind propulsion parameters between nodes that the UAV passes through; In the absence of wind, The drone flew through the node to the The average speed of each drone passing through the node; Indicates the The location coordinates of the nodes that the drone passes through; Indicates the The location coordinates of the nodes that the drone passes through; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates the drone is at position ,time wind force; Indicates the drone is at position ,time wind direction; represents the inverse tangent function; Indicates the The position coordinates of the nodes that the UAV passes through are The Euclidean distance between the position coordinates of the nodes passed by the drone.

[0029] In the wind propulsion parameter calculation formula, when a UAV flies between two UAV passing nodes, the angle between the wind direction and the UAV will change with the change of time scale and space scale. Similarly, the magnitude of the wind will also change with the change of the two scales. Therefore, the wind force , and the angle between the wind direction and the drone It's all about spatial scale and time scale function, so we get and If the angle between the wind direction and the flight direction of the drone is less than 90°, it means that the wind has a driving effect on the drone, and the stronger the wind, the stronger the driving effect. On the contrary, if the angle between the wind direction and the flight direction of the drone is greater than 90°, it means that the wind has an obstruction effect on the flight of the drone, and the stronger the wind, the stronger the obstruction effect. The bigger the wind is, the stronger the propulsion effect of the wind is. The drone passes through the node Double integration of all wind forces and wind directions between the nodes that the UAV passes through is obtained , to indicate the The drone passes through the node The wind force between the nodes that the drone passes through has a propulsive effect on the drone; in the absence of wind, The drone flew through the node to the The average speed of the drone passing through the node The faster the Euclidean distance between two nodes The smaller the The drone passes through the node The flight time between nodes that the UAV passes through The smaller the value, the higher the flight efficiency of the drone in the windless state. The wind force’s propulsion effect on the drone is used as the weight. The larger the weight, the greater the wind force propulsion parameter. If the weight is less than 0, it means that the wind force has an obstructive effect on the drone. In order to keep the result positive in the subsequent process, the result of the integration is put into the function In the process, ensure that the subsequent results are positive numbers. The larger the result, the stronger the beneficial effect on the drone.

[0030] It should be noted that the UAV The drone passes through the node The average speed between the nodes that the drone passes through, The drone passes through the node The corresponding time when each drone passes through a node can be obtained through the drone's own system and will not be described in detail here.

[0031] In addition to the impact of wind on drones, the height of trees in different areas also has a certain impact on drone power consumption. Because drones need to avoid trees when flying in forests, they must fly above the height of the bushes to prevent irreversible damage to the drone. Since drones fly at different altitudes, their power consumption varies. Therefore, it is necessary to calculate the energy loss caused by the altitude of the trees when flying between nodes. This involves analyzing the energy consumption parameters between each two adjacent nodes along the reference path.

[0032] Preferably, in one embodiment of the present invention, the method for obtaining air consumption parameters includes: obtaining all preset flight altitude gear types divided according to the height requirements of the vegetation below when the drone flies in the forest. The flight altitude gear type is the flight altitude required when the drone encounters different types of trees. The specific content is a technical means well known to those skilled in the art and will not be elaborated here.

[0033] Obtaining the power consumption and flight time of the drone at each flight altitude and gear type can be directly obtained through the drone's background system, which will not be described in detail here.

[0034] First, the flight altitude gear type between each two adjacent drone passing nodes is obtained, and the drone power consumption of each flight altitude gear type between each two adjacent drone passing nodes is multiplied by the flight time, and the sum is accumulated to obtain the drone's idle power parameters between each two adjacent drone passing nodes.

[0035] The comprehensive evaluation parameters of the reference path are obtained based on the wind propulsion parameters, UAV air consumption parameters, distance distribution, UAV flight time and UAV power consumption between each two adjacent UAV passing nodes in the reference path.

[0036] Preferably, in one embodiment of the present invention, the method for obtaining the comprehensive evaluation parameter includes: obtaining the comprehensive evaluation parameter according to a comprehensive evaluation parameter calculation formula, the comprehensive evaluation parameter calculation formula being as follows: Where, Indicates the first The drone passes through the node Comprehensive evaluation parameters between nodes that UAVs pass through; Indicates the first The drone passes through the node Wind propulsion parameters between nodes that the UAV passes through; Indicates that the drone is at the same altitude. The drone passes through the node to the The average power consumed by each drone passing through a node; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates that the drone is The drone passes through the node to the The idle consumption parameters of the nodes that the UAV passes through; Indicates the The location coordinates of the nodes that the drone passes through; Indicates the The location coordinates of the nodes that the drone passes through; Indicates the maximum distance between every two adjacent nodes in the reference path; Indicates the number of nodes that the drone passes through; Represents an exponential function with a natural constant as its base.

[0037] In the comprehensive evaluation parameter calculation formula, since the power consumption of the UAV is different under different wind forces, the power consumption of the UAV is different under different wind forces. The drone passes through the node The product of the average power consumption and flight time of the drones between the nodes that the drones pass through is taken as the The drone passes through the node The power consumption of the drones between the nodes that the drones pass through; and The drone passes through the node The larger the ratio of wind propulsion parameters to the power consumption of the drone between the nodes it passes through, the greater the The drone passes through the node The smaller the air consumption parameter is, the smaller the power consumption of the drone between the two nodes is, and the smaller the Euclidean distance between the two nodes is, the more reasonable the path planning between the two nodes is, and the more efficient the utilization of the air consumption parameter is. Controls the value range.

[0038] Step S3: In the DQN algorithm, the comprehensive evaluation parameters are used for path planning.

[0039] In one embodiment of the present invention, in the DQN algorithm, path planning is performed based on comprehensive evaluation parameters, including: when using the DQN algorithm to plan the path of a drone, the state data of the drone at different spatial locations is input, such as the drone's position coordinate information, surrounding environment information, wind information, energy consumption information, etc. It should be noted that the above information can be obtained from the drone's system parameters and is not described in detail here. When the drone is at a certain drone path node, the position coordinate information of the drone path node, the surrounding environment information, and the comprehensive evaluation parameters of the drone path node and the next adjacent drone path node in all drone paths are input into the DQN algorithm. These parameters are used to generate a corresponding Q value, and the next adjacent drone path node with the largest Q value is used as the drone path node to be reached next. This process is repeated until all drone path nodes that the drone has passed through are selected, thereby completing the drone path planning. It should be noted that the DQN algorithm is a technical means well known to those skilled in the art and is not described in detail here.

[0040] In summary, all fire-prone areas in the forest are obtained, and each fire-prone area is used as a UAV path node; the reference planning radius of each UAV path node is calculated based on the distance distribution between all UAV path nodes; the paths of all UAV path nodes are designed based on the reference planning radius to obtain all UAV paths; any UAV path is selected as the reference path; the wind propulsion parameters between each two adjacent UAV path nodes in the reference path are obtained based on the distance between each two adjacent UAV path nodes in the reference path, the UAV flight speed, the UAV flight time and the wind characteristics; the air consumption parameters between each two adjacent UAV path nodes in the reference path are obtained based on the UAV flight altitude, the UAV flight time and the UAV power consumption between each two adjacent UAV path nodes in the reference path; the comprehensive evaluation parameters between each two adjacent UAV path nodes in the reference path are obtained based on the wind propulsion parameters, the UAV air consumption parameters, the distance distribution, the UAV flight time and the UAV power consumption between each two adjacent UAV path nodes in the reference path; in the DQN algorithm, the comprehensive evaluation parameters are used for path planning.

[0041] One embodiment of the present invention provides a UAV path planning system based on reinforcement learning, the system includes a memory, a processor and a computer program, wherein the memory is used to store the corresponding computer program, the processor is used to run the corresponding computer program, and the computer program can implement the method described in steps S1-S3 when running in the processor, specifically as follows Figure 2 As shown, it includes: a node acquisition module 101, which is used to obtain all fire-prone areas in the forest and use each fire-prone area as a node passed by the drone.

[0042] The UAV path analysis module 102 is used to calculate the reference planning radius of each UAV path node based on the distance distribution between all UAV path nodes; design paths for all UAV path nodes based on the reference planning radius to obtain all UAV paths; select any UAV path as a reference path; obtain the wind propulsion parameters between each two adjacent UAV path nodes in the reference path based on the distance between each two adjacent UAV path nodes in the reference path, the UAV flight speed, the UAV flight time and the wind characteristics; obtain the air consumption parameters between each two adjacent UAV path nodes in the reference path based on the UAV flight altitude, the UAV flight time and the UAV power consumption between each two adjacent UAV path nodes in the reference path; obtain the comprehensive evaluation parameters between each two adjacent UAV path nodes in the reference path based on the wind propulsion parameters, the UAV air consumption parameters, the distance distribution, the UAV flight time and the UAV power consumption between each two adjacent UAV path nodes in the reference path.

[0043] The path planning module 103 is used to perform path planning using comprehensive evaluation parameters in the DQN algorithm.

[0044] The third object of an embodiment of the present invention is to provide a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method described in steps S1-S3 is implemented when the processor executes the computer program.

[0045] A fourth object of an embodiment of the present invention is to provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in steps S1-S3 is implemented.

[0046] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0047] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A UAV path planning method based on reinforcement learning, characterized in that: The method includes: obtaining all high-incidence areas of fire in a forest, and taking each of the high-incidence areas as a UAV path node; calculating a reference planning radius of each UAV path node based on the distance distribution between each pair of all UAV path nodes; performing traversal path design on all UAV path nodes based on the reference planning radius to obtain a set of all UAV paths; selecting any UAV path as a reference path; obtaining a wind propulsion parameter between each two adjacent UAV path nodes in the reference path based on the distance between each two adjacent UAV path nodes in the reference path, the UAV flight speed, the UAV flight time, and the wind characteristics; obtaining an idle consumption parameter between each two adjacent UAV path nodes in the reference path based on the UAV flight altitude, the UAV flight time, and the UAV power consumption between each two adjacent UAV path nodes in the reference path; obtaining a comprehensive evaluation parameter between each two adjacent UAV path nodes in the reference path based on the wind propulsion parameter, the idle consumption parameter of the UAV, the distance distribution, the UAV flight time, and the UAV power consumption between each two adjacent UAV path nodes in the reference path; and using the comprehensive evaluation parameter to perform path planning in a DQN algorithm.

2. The UAV path planning method based on reinforcement learning according to claim 1, characterized in that: The method for obtaining the reference planning radius includes: obtaining the reference planning radius according to a reference planning radius calculation formula, and the reference planning radius calculation formula is as follows: Where, Indicates the reference planning radius of each node that the UAV passes through; Indicates the number of nodes that the drone passes through, greater than 1; Indicates that each drone passes through the node The distance between the nodes passed by other drones, is a natural number representing the number of drones.

3. The UAV path planning method based on reinforcement learning according to claim 1, characterized in that: According to the reference planning radius, traversal paths are designed for all drone passing nodes to obtain a set of all traversal paths, including: randomly selecting one drone passing node as a reference node; taking the reference node as the starting node, the starting node as the center of the circle, and the reference planning radius as the radius to construct a reference planning circle of the starting node, selecting each drone passing node in the reference planning circle as the next connection node of the drone, taking any connection node as the new starting node, repeating the above steps until there are no more drone passing nodes in the reference planning circle, connecting the center of the current reference planning circle with the drone passing node that has not appeared in the generated path and is closest to the starting node as the latest connection node, taking the latest connection node as the new starting node, repeating the above steps until the drone passes through all drone passing nodes, and obtaining all drone paths with the reference node as the starting node; traversing all drone passing nodes to obtain all drone paths with each drone passing node as the starting node.

4. The UAV path planning method based on reinforcement learning according to claim 1, characterized in that: The method for obtaining the wind propulsion parameter includes: obtaining the wind propulsion parameter according to a wind propulsion parameter calculation formula, and the wind propulsion parameter calculation formula is as follows: Where, Indicates the first The drone passes through the node Wind propulsion parameters between nodes that the UAV passes through; In the absence of wind, The drone flew through the node to the The average speed of each drone passing through the node; Indicates the The location coordinates of the nodes that the drone passes through; Indicates the The location coordinates of the nodes that the drone passes through; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates the drone is at position ,time wind force; Indicates the drone is at position ,time wind direction; represents the inverse tangent function; Indicates the The position coordinates of the nodes that the UAV passes through are The Euclidean distance between the position coordinates of the nodes passed by the drone.

5. The UAV path planning method based on reinforcement learning according to claim 1, characterized in that: The method for obtaining the air consumption parameters includes: obtaining all preset flight altitude gear types divided according to the height requirements of the vegetation below when the drone flies in the forest; obtaining the drone power consumption and flight time of the drone in each flight altitude gear type; multiplying the drone power consumption and flight time of each flight altitude gear type between each two adjacent drone passing nodes, and accumulating the sum to obtain the air consumption parameters of the drone between each two adjacent drone passing nodes.

6. The UAV path planning method based on reinforcement learning according to claim 1, characterized in that: The method for obtaining the comprehensive evaluation parameter includes: obtaining the comprehensive evaluation parameter according to a comprehensive evaluation parameter calculation formula, and the comprehensive evaluation parameter calculation formula is as follows: Where, Indicates the first The drone passes through the node Comprehensive evaluation parameters between nodes that UAVs pass through; Indicates the first The drone passes through the node Wind propulsion parameters between nodes that the UAV passes through; Indicates that the drone is at the same altitude. The drone passes through the node to the The average power consumed by each drone passing through a node; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates that the drone has arrived at The time when a drone passes through a node; Indicates that the drone is The drone passes through the node to the The idle consumption parameters of the nodes that the UAV passes through; Indicates the The location coordinates of the nodes that the drone passes through; Indicates the The location coordinates of the nodes that the drone passes through; Indicates the maximum distance between every two adjacent nodes in the reference path; Indicates the number of nodes that the drone passes through; Represents an exponential function with a natural constant as its base.

7. A UAV path planning system based on reinforcement learning, the system comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the UAV path planning method based on reinforcement learning as described in any one of claims 1 to 6 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of a drone path planning method based on reinforcement learning as described in any one of claims 1 to 6 are implemented.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the UAV path planning method based on reinforcement learning as described in any one of claims 1 to 6 are implemented.

Images