Progressive map prediction method for unknown environment exploration of unmanned aerial vehicle

Abstract

The invention discloses a progressive map prediction method for unknown environment exploration of an unmanned aerial vehicle. The method comprises the following steps: acquiring an initial digital elevation map, and normalizing the initial digital elevation map to obtain a target digital elevation map; dividing the target digital elevation map by using the threshold height to obtain a planar two-dimensional map corresponding to the target digital elevation map; determining an initial flying point of the unmanned aerial vehicle based on the planar two-dimensional map and the initial flying point; based on the target navigational speed, the gravitational acceleration, the maximum allowable normal overload coefficient and the target time interval of the unmanned aerial vehicle at the initial flying spot, the target turning radius, the unmanned aerial vehicle motion step length and the target course angle of the unmanned aerial vehicle in the planar two-dimensional map are determined. According to the invention, the technical problem that the accuracy of the explored map is low because the unmanned aerial vehicle explores the map autonomously through deep reinforcement learning in an unknown environment in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of map exploration technology, and in particular to a progressive map prediction method for exploring unknown environments of unmanned aerial vehicles. Background Art

[0002] In complex unknown environments, the autonomous exploration capability of drones is particularly important, especially in the mountains, plateaus, river valleys, deserts and other areas in western China. There are relatively mature inventions at home and abroad for drone trajectory planning in known environments, but drone exploration and path planning in unknown environments still face challenges, mainly focusing on how to achieve efficient and complete exploration under the condition of lack of global environmental information; At present, domestic and foreign inventions are mainly based on boundary-driven autonomous exploration. In recent years, more and more scholars have adopted new methods such as deep reinforcement learning for exploration and invention, which has promoted the rapid development of the field of unknown environment exploration; but whether it is boundary-driven autonomous exploration or deep reinforcement learning-based autonomous exploration, it is impossible to build a complete global map. Summary of the invention

[0003] The embodiment of the present invention provides a progressive map prediction method for unmanned aerial vehicle exploration of unknown environments, so as to at least solve the technical problem in the prior art that unmanned aerial vehicles explore maps in unknown environments through autonomous exploration through deep reinforcement learning, and the accuracy of the explored maps is low.

[0004] According to one aspect of an embodiment of the present invention, a progressive map prediction method for exploring an unknown environment of a drone is provided. The method may include: obtaining an initial digital elevation map, normalizing the initial digital elevation map, and obtaining a target digital elevation map; dividing the target digital elevation map by a threshold height to obtain a planar two-dimensional map corresponding to the target digital elevation map; determining the initial flyable point of the drone based on the planar two-dimensional map and the initial flight point; determining the target turning radius, the drone motion step length, and the target heading angle of the drone in the planar two-dimensional map based on the target speed, gravity acceleration, the maximum allowable normal overload factor, and the target time interval of the drone at the initial flyable point; determining the exploration map explored by the drone in an unknown environment based on the target turning radius, the drone motion step length, and the target heading angle of the drone; constructing a target neural network; inputting the exploration map explored by the drone in an unknown environment into the first convolution layer to obtain a feature map of the exploration map; inputting the feature map of the exploration map into several transform modules to obtain a feature vector of the exploration map; inputting the feature vector of the exploration map into the second convolution layer to obtain a prediction map corresponding to the initial digital elevation map.

[0005] Optionally, the initial digital elevation map is normalized to obtain a target digital elevation map expression as follows:

[0006] in, is the pixel value of each point in the target digital elevation map, is the pixel value of each point in the initial digital elevation map, is the pixel value of the lowest point in the initial digital elevation map, It is the pixel value of the highest point in the initial digital elevation map.

[0007] Optionally, the method of dividing the target digital elevation map by using a threshold height to obtain a flat two-dimensional map corresponding to the target digital elevation map includes: when a pixel value corresponding to a pixel point in the target digital elevation map is greater than a threshold height, setting the pixel value of the pixel point in the target digital elevation map to one; when a pixel value corresponding to a pixel point in the target digital elevation map is less than a threshold height, setting the pixel value of the pixel point in the target digital elevation map to zero; and obtaining a flat two-dimensional map based on the pixel values ​​zero and one of the pixel points.

[0008] Optionally, determining the initial flyable point of the UAV based on the planar two-dimensional map and the initial flight point includes: determining candidate flight points of the UAV based on the planar two-dimensional map and the initial flight point; and taking any point of the candidate flight points of the UAV as the initial flyable point of the UAV.

[0009] Optionally, the method determines the target turning radius, movement step length and target heading angle of the UAV on the flat two-dimensional map based on the target speed of the UAV at the initial flyable point, the acceleration of gravity, the maximum allowable normal overload coefficient of the UAV and the target time interval, including: determining the target turning radius based on the target speed of the UAV at the initial flyable point, the acceleration of gravity and the maximum allowable normal overload coefficient of the UAV; determining the movement step length of the UAV based on the target speed and target time interval of the UAV at the initial flyable point; determining the target heading angle based on the movement step length and the target turning radius.

[0010] Optionally, the expression of the target turning radius is:

[0011] Where g is the acceleration due to gravity, is the maximum allowable normal overload factor of the UAV, is the target speed of the UAV, is the target turning radius.

[0012] Optionally, the expression of the UAV motion step length is:

[0013] in, is the motion step length of the drone, is the target time interval.

[0014] Optionally, the target heading angle is expressed as:

[0015] in, is the target heading angle.

[0016] Optionally, determining the exploration map explored by the UAV in an unknown environment based on the target turning radius of the UAV, the motion step length of the UAV and the target heading angle includes: determining the position, navigation direction and yaw angle of the UAV in the two-dimensional map at each moment based on the target turning radius of the UAV, the motion step length of the UAV and the target heading angle; determining the exploration map explored by the UAV in the unknown environment based on the position, navigation direction and yaw angle in the two-dimensional map at each moment.

[0017] Beneficial effects of the present invention: The present invention proposes a progressive map prediction method for UAV exploration of unknown environments to meet the exploration and navigation needs of UAVs in unknown and complex environments. Traditional path planning methods usually rely on pre-built global maps. It is difficult to complete path planning and obstacle avoidance tasks when the global map information is incomplete. The progressive map prediction method obtains local environmental data in real time, gradually builds and optimizes the map, thereby realizing autonomous planning of UAV paths. The goal of this invention is not only to improve the autonomous navigation capability of UAVs, but also to improve their adaptability and robustness in complex environments. By introducing deep learning and reinforcement learning technologies, this method can continuously optimize path planning decisions during flight, thereby improving task completion efficiency. In theory, this invention can provide new methodological support for the field of UAV exploration path planning; in practical applications, this method can be widely used in tasks such as disaster relief, resource exploration and environmental monitoring, to improve the autonomous exploration capability and task execution efficiency of UAVs. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings: Figure 1 is a flow chart of a progressive map prediction method for exploring an unknown environment of a UAV according to an embodiment of the present invention; Figure 2 is a schematic diagram of a target digital elevation map according to an embodiment of the present invention; Figure 3 is a schematic diagram of a planar two-dimensional map determination process according to an embodiment of the present invention; Figure 4 is a schematic diagram of autonomous environment exploration of a drone according to an embodiment of the present invention; Figure 5 is a schematic diagram of the structure of several Transformer modules according to an embodiment of the present invention; Figure 6 is a schematic diagram of dynamic mask update according to an embodiment of the present invention; Figure 7 is a schematic diagram of map recovery according to an embodiment of the present invention; Figure 8 is a schematic diagram of an initial flight point and an initial non-flyable point according to an embodiment of the present invention; Fig. 9 is a schematic diagram of discrete and continuous action space modeling according to an embodiment of the present invention; Fig.10 Schematic diagram of a reward function according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] Example 1 According to an embodiment of the present invention, a progressive map prediction method for exploring an unknown environment for a UAV is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system comprising at least one set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0022] Figure 1 is a flowchart of a progressive map prediction method for exploring unknown environments of unmanned aerial vehicles according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps: Step S101, obtaining an initial digital elevation map, normalizing the initial digital elevation map, and obtaining a target digital elevation map.

[0023] In the technical solution provided in the above step S101 of the present invention, Figure 2 is a schematic diagram of a target digital elevation map according to an embodiment of the present invention, Figure 2 (a) is the main view of the target digital elevation map. Figure 2 (b) is a bird's-eye view of the target digital elevation map. The initial digital elevation map is a digital map that stores surface elevation information in the form of raster data. This elevation information can be used for drawing topographic maps, conducting geographic information system analysis, and other geographical and environmental science applications. The initial digital elevation map is normalized and mapped within [0, 1], thus obtaining the target digital elevation map.

[0024] Step S102, dividing the target digital elevation map by using the threshold height to obtain a planar two-dimensional map corresponding to the target digital elevation map.

[0025] In the technical solution provided in the above step S102 of the present invention, Figure 3 is a schematic diagram of a planar two-dimensional map determination process according to an embodiment of the present invention, Figure 3 (a) is to divide the target digital elevation map using the threshold height. Figure 3 (b) is a flat two-dimensional map, Figure 3 The purple area in (b) is the initial area where the drone can fly, and the red area is the obstacle area.

[0026] Step S103, determining the initial flyable point of the UAV based on the planar two-dimensional map and the initial flight point.

[0027] In the technical solution provided in the above step S103 of the present invention, the initial flyable point of the UAV is determined according to the planar two-dimensional map and the initial flight point.

[0028] Step S104, based on the target speed of the UAV at the initial flyable point, gravity acceleration, the maximum allowable normal overload factor of the UAV and the target time interval, determine the target turning radius of the UAV on the flat two-dimensional map, the UAV movement step length and the target heading angle.

[0029] In the technical solution provided in the above step S104 of the present invention, the target speed of the UAV at the initial flyable point, the gravitational acceleration, the maximum allowable normal overload coefficient of the UAV and the target time interval are calculated to obtain the target turning radius of the UAV on the flat two-dimensional map, the UAV movement step length and the target heading angle.

[0030] Step S105, determining an exploration map explored by the drone in an unknown environment based on the target turning radius of the drone, the motion step length of the drone, and the target heading angle.

[0031] In the technical solution provided in the above step S105 of the present invention, an exploration map explored by the drone in an unknown environment is obtained according to the target turning radius of the drone, the movement step length of the drone and the target heading angle. Figure 4 is a schematic diagram of autonomous environment exploration of a drone according to an embodiment of the present invention, Figure 4 (a) is the global map, Figure 4 (a) The 512-pixel image generated after random segmentation of the 12.5-meter digital elevation map of the target area 512 pixel global map, Figure 4 (b) To explore the map, since the global map is unknown to the drone, a mask is applied to it. Whenever the drone explores a new area, the exploration map is updated. After the drone is initialized at the starting point, it randomly selects a heading angle and starts to explore at a constant speed until the end or the maximum number of steps is reached, at which point the current round of exploration is stopped. Figure 4 (c) is background replacement, that is, Figure 4 (b) replaces the background in the exploration map. Figure 4 (d) is the mask map, Figure 4 (c) Perform the conversion.

[0032] Step S106, constructing a target neural network.

[0033] In the technical solution provided in the above step S106 of the present invention, a target convolutional neural network is constructed, wherein the target neural network is obtained by removing the normalized layers in the Transformer in the prior art and replacing the residual connection with feature concatenation. Figure 5 is a schematic diagram of the structure of several Transformer modules according to an embodiment of the present invention, Figure 5 It can be seen that the normalization of the middle layer of the Transformer in the prior art is removed, and the residual connection is replaced by feature concatenation.

[0034] Step S107, inputting the exploration map explored by the drone in the unknown environment into the first convolutional layer to obtain a feature map of the exploration map.

[0035] In the technical solution provided in the above step S107 of the present invention, the exploration map explored by the drone in the unknown environment is input into the first convolutional layer to obtain a feature map of the exploration map.

[0036] Step S108, inputting the feature map of the exploration map into several transform modules to obtain the feature vector of the exploration map.

[0037] In the technical solution provided in the above step S108 of the present invention, the expression of the process of inputting the feature map of the exploration map into the multi-head context attention mechanism in the first transform module is:

[0038] in, is a dynamic mask matrix, which is used to indicate valid and invalid tags, and controls the weight of invalid tags to be close to 0, so as to optimize the computing performance and effect. Q is the query matrix, which is used to represent the "query" mode of the current input when focusing on features at different positions. K is the key matrix, which represents the "key" feature of each position in the input feature, and is used to match the query feature. is the transpose of K, V is the value matrix, which represents the input content information, is the dimension of the key vector K, which is used to scale the value of the dot product result to prevent the value from being too large and causing Softmax output bias; Figure 6 is a schematic diagram of dynamic mask update according to an embodiment of the present invention, such as Figure 6 As shown in the figure, the feature map of the exploration map is initially divided into a 2×2 window, "U" represents the update mask after attention, and "S" represents the window movement; the initial mask of the model is generated based on the input mask. With the multiple iterative updates of the attention mechanism, the mask gradually expands the effective area, significantly enhancing the utilization of features in the repair process. The feature map of the exploration map is processed by several transform modules to obtain the feature vector of the exploration map.

[0039] Step S109, inputting the feature vector of the exploration map into the second convolutional layer to obtain a predicted map corresponding to the initial digital elevation map.

[0040] In the technical solution provided in the above step S109 of the present invention, Figure 7 is a schematic diagram of map recovery according to an embodiment of the present invention, Figure 7 The first line in is the original map, that is Figure 4 (a) is the global map, the second line is the recovered map, that is, the predicted map, and the last line is the mask map corresponding to the predicted map of the drone in the location environment.

[0041] The above method of this embodiment is further introduced below.

[0042] As an optional implementation mode, in step S101, the initial digital elevation map is normalized to obtain the target digital elevation map expression as follows:

[0043] in, is the pixel value of each point in the target digital elevation map, is the pixel value of each point in the initial digital elevation map, is the pixel value of the lowest point in the initial digital elevation map, It is the pixel value of the highest point in the initial digital elevation map.

[0044] As an optional implementation method, the target digital elevation map is divided by using a threshold height to obtain a flat two-dimensional map corresponding to the target digital elevation map, including: when a pixel value corresponding to a pixel point in the target digital elevation map is greater than a threshold height, the pixel value of the pixel point in the target digital elevation map is set to one; when a pixel value corresponding to a pixel point in the target digital elevation map is less than a threshold height, the pixel value of the pixel point in the target digital elevation map is set to zero; based on the pixel values ​​zero and one of the pixel points, a flat two-dimensional map is obtained.

[0045] In this embodiment, if Figure 3 As shown, if the pixel value corresponding to the pixel point in the target digital elevation map is greater than the threshold height, the pixel value of the pixel point in the target digital elevation map is set to one; if the pixel value corresponding to the pixel point in the target digital elevation map is less than the threshold height, the pixel value of the pixel point in the target digital elevation map is set to zero; according to the pixel values ​​zero and one of the pixel points, a two-dimensional plane is obtained.

[0046] As an optional implementation method, the method of determining the initial flyable point of the drone based on the planar two-dimensional map and the initial flight point includes: determining the candidate flight points of the drone based on the planar two-dimensional map and the initial flight point; and taking any point of the candidate flight points of the drone as the initial flyable point of the drone.

[0047] In this embodiment, Figure 3 In (b), the points in the purple area are the initial flight points of the drone. However, since some purple areas are closer to the red obstacle areas, even if the drone is initialized at these points, there is a high probability that the drone will crash in the early stages of exploration. Therefore, it is necessary to perform obstacle detection on the environment around the initial flight point before initialization to ensure that the drone starts exploring in a stable and safe environment. The specific detection method is to set the initial generation radius, take the initial flight point as the center, and intercept the edge length of The square matrix is ​​denoted as S, and point-by-point product is performed with a matrix of the same dimension. The element values ​​of the S matrix meet the following conditions for safe takeoff:

[0048] Among them, i and j are the horizontal and vertical coordinates of the pixel point in the current UAV plane two-dimensional map; by calculating the product result of the matrix S, if the maximum value does not exceed 0, it means that there are no obstacles near the candidate point and it is suitable as an initial flight point. It is added to the candidate point queue for random sampling; otherwise, it is considered that there are dense obstacles near the point and it is not suitable as an initial flight point. Figure 8 is a schematic diagram of the initial flight point and the initial non-flyable point according to an embodiment of the present invention, wherein the inner circle of the ring represents the field of view observable by the drone, and the outer circle is the initial detection radius.

[0049] As an optional implementation method, the target turning radius, movement step length and target heading angle of the UAV on a two-dimensional map are determined based on the target speed of the UAV at the initial flyable point, the acceleration of gravity, the maximum allowable normal overload coefficient of the UAV and the target time interval. The method includes: determining the target turning radius based on the target speed of the UAV at the initial flyable point, the acceleration of gravity and the maximum allowable normal overload coefficient of the UAV; determining the movement step length of the UAV based on the target speed and target time interval of the UAV at the initial flyable point; and determining the target heading angle based on the movement step length and the target turning radius.

[0050] In this embodiment, the target speed of the UAV at the initial flyable point, the gravitational acceleration, and the maximum allowable normal overload coefficient of the UAV are calculated to obtain the target turning radius; the target speed and target time interval of the UAV at the initial flyable point are calculated to obtain the UAV movement step length; the UAV movement step length and the target turning radius are calculated to obtain the target heading angle.

[0051] As an optional implementation manner, the expression of the target turning radius is:

[0052] Where g is the acceleration due to gravity, is the maximum allowable normal overload factor of the UAV, is the target speed of the UAV, is the target turning radius.

[0053] In this embodiment, the maximum allowable normal overload coefficient of the UAV, the target speed and the gravitational acceleration are substituted into the expression of the target turning radius to obtain the target turning radius, where g is the gravitational acceleration and is taken as 9.82. , The maximum allowable normal overload factor of the drone is 20, set The speed is 20m / s. The minimum turning radius (target turning radius) of the drone model can be obtained by calculation. About 2.04m.

[0054] As an optional embodiment, the expression of the UAV motion step length is:

[0055] in, is the motion step length of the drone, is the target time interval.

[0056] In this embodiment, the target time interval and the target speed of the drone are substituted into the expression of the drone motion step length to obtain the drone motion step length, where the target time interval is 0.1s. is 2m.

[0057] As an optional implementation mode, the expression of the target heading angle is:

[0058] in, is the target heading angle.

[0059] In this embodiment, the target turning radius and the UAV motion step length are substituted into the expression of the target heading angle to obtain the target heading angle, where: is 29.3°, so the maximum heading angle of the drone is in the range of [-29.3°, 29.3°].

[0060] As an optional implementation method, the method of determining the exploration map explored by the UAV in an unknown environment based on the UAV's target turning radius, UAV movement step length and target heading angle includes: determining the UAV's position, navigation direction and yaw angle in the two-dimensional map at each moment based on the UAV's target turning radius, UAV movement step length and target heading angle; determining the exploration map explored by the UAV in an unknown environment based on the position, navigation direction and yaw angle in the two-dimensional map at each moment.

[0061] In this embodiment, the update of the position and direction of the next step of the autonomous exploration planning of the drone is shown as follows:

[0062] in, are the current and next moment positions of the drone, respectively. , are the direction of the drone at the current moment and the direction at the next moment respectively. is the yaw angle.

[0063] The system of the present invention uses the same action type internally, and does not involve mixed actions. After determining the maximum heading angle range of the drone, the set of its action space is also determined, and the values ​​of all actions are within the heading angle range. The design of the action space takes into account the relative heading angle of the drone, and establishes a discrete action space and a continuous action space for it. Fig. 9 is a schematic diagram of discrete and continuous action space modeling according to an embodiment of the present invention, such as Fig. 9 As shown in the figure, the discrete action is to equally discretize the maximum heading angle range of the drone, and finally obtain a finite countable action set, which can reduce the dimension of the action space; the continuous action is to map the maximum heading angle range of the drone to [-1, 1] through the tanh activation function, so that the action space is continuous, and finally obtain an infinite real number action set, which has higher complexity but stronger adaptability to the environment. Fig. 9 (b) Continuous action.

[0064] In reinforcement learning, the reward function plays a pivotal role and is the main guide for the learning and decision-making of the agent. The reward function defines the goal of the problem. By giving the agent an immediate reward after taking a certain action at each time step, it guides the agent to find a strategy that can maximize the cumulative reward. In this invention, in order to ensure that the drone can explore as many unknown areas as possible, the first thing is to let the drone learn an autonomous obstacle avoidance strategy to prevent crashes. The reward function designed based on this is shown in the following formula:

[0065] When the drone collides with an obstacle, the reward value is a large negative value of -1000, and the exploration of this round will end directly. This state is called completion in reinforcement learning. When the drone is in a normal exploration state, its reward is The area of ​​the unknown area newly explored in a single step is multiplied by a fixed coefficient. Taking into account the physical limitations of the drone's endurance, in addition to ending the exploration when it hits an obstacle, it is also set that the exploration will end when the drone's exploration steps per round reach the maximum number.

[0066] Fig.10 is a schematic diagram of a reward function according to an embodiment of the present invention, such as Fig.10 As shown in the figure, the reward curve obtained by initially training the reinforcement learning agent. In this experiment, the maximum number of exploration steps in each episode is 1000, and the final reward curve tends to converge.

[0067] In order to further implement the reward function, the objective function of PPO-Clip is proposed. The expression of the objective function of PPO-Clip is shown as follows:

[0068]

[0069] in, represents the weight parameters of the network, To explore based on current strategies The obtained trajectory, It was the previous strategy. Yes Status Take action The advantage function of is importance sampling; clipping function Will Truncated at [ ], that is, when the function represents the relative change of the new strategy to the old strategy exceeds Output , less than It will output , in order to ensure the similarity between the new and old strategies; when the advantage function value is greater than 0, it means that the state-action pair is more adaptable in the environment and helps to obtain greater rewards, that is, The bigger the better, the larger the objective function value can be obtained; but When more than When , it does not improve the training effect of the model, nor is it conducive to the learning of the optimal strategy, and vice versa; finally, the smaller of the truncated objective function and the untruncated objective function is taken as the final objective function; it is this conservative update method that greatly improves the stability and convergence of model training.

[0070] The importance sampling ratio With advantage function When multiplying, we are actually adjusting the magnitude of the policy update so that the policy can be updated efficiently and instability can be avoided. The specific process is as follows: a.When >0 (good move) >1 If the current policy is more likely to choose ,So will be greater than 1, after multiplication, >1 will further encourage the selection of this good action , thereby increasing the probability that the strategy will choose this action.

[0071] <1 If the current strategy underestimates the old strategy ,Right now Less than 1, indicating that the current strategy is for the action The probability of selection is low. After multiplication, <1 will compensate for it and improve the action The selection probability makes the strategy more inclined to choose this advantageous action.

[0072] b. When <0(differential action) >1 If the current policy is more likely to choose ,Right now >1, then after multiplication, >1 will enhance punishment, further reducing the probability of choosing this bad action.

[0073] <1 If the current policy is less likely to choose an action relative to the old policy ,Right now <1, then Will act The degree of weakening is adjusted to reduce the probability of selecting this differential action.

[0074] The effect of multiplication: The multiplication of the importance sampling ratio and the advantage function actually controls the amplitude of the strategy update: When action is a good action (A>0), and the current policy tends to choose it ( >1), further encourages the selection of this action: when the current policy underestimates it ( <1) and compensate for it.

[0075] When action is a bad action (A<0), and the current policy tends to choose it ( >1), strengthen the penalty and reduce the probability of choosing this action; if the current strategy does not choose it ( <1), and further adjust the selection probability of the action by multiplying it to avoid excessive punishment.

[0076] , that is, the current strategy ( ) relative to the old strategy ( ) ratio.

[0077] when >1, the current strategy is more inclined to choose actions than the old strategy .

[0078] when <1, the current strategy is less inclined to choose actions than the old strategy .

[0079] In an embodiment of the present invention, an initial digital elevation map is obtained and normalized to obtain a target digital elevation map; the target digital elevation map is divided by a threshold height to obtain a planar two-dimensional map corresponding to the target digital elevation map; an initial flyable point of the UAV is determined based on the planar two-dimensional map and the initial flight point; a target turning radius, a motion step length and a target heading angle of the UAV in the planar two-dimensional map are determined based on the target speed, gravity acceleration, a maximum allowable normal overload factor and a target time interval of the UAV at the initial flyable point; and a target turning radius, a motion step length and a target heading angle of the UAV in the planar two-dimensional map are determined based on the target turning radius, the motion step length and the target heading angle of the UAV in the unknown The method comprises the following steps: first, inputting the exploration map explored by the UAV in the unknown environment into the first convolutional layer to obtain the feature map of the exploration map; second, inputting the feature map of the exploration map into several transform modules to obtain the feature vector of the exploration map; third, inputting the feature vector of the exploration map into the second convolutional layer to obtain the prediction map corresponding to the initial digital elevation map, which solves the technical problem that the UAV explores the map in the unknown environment through autonomous exploration through deep reinforcement learning in the prior art, and the accuracy of the explored map is low, and achieves the technical effect that the UAV explores the map in the unknown environment through multiple strategies to make the explored map accurate.

[0080] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0081] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0083] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0084] In addition, each functional unit in each embodiment of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0085] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A progressive map prediction method for exploring unknown environments of unmanned aerial vehicles, characterized in that: include: Acquire an initial digital elevation map, perform normalization processing on the initial digital elevation map, and obtain a target digital elevation map; The target digital elevation map is divided by using the threshold height to obtain a planar two-dimensional map corresponding to the target digital elevation map; Based on the two-dimensional map and the initial flight point, determine the initial flyable point of the drone; Based on the target speed of the UAV at the initial flyable point, gravity acceleration, the maximum allowable normal overload factor of the UAV and the target time interval, the target turning radius of the UAV on the flat two-dimensional map, the UAV movement step length and the target heading angle are determined; Determine the exploration map explored by the drone in an unknown environment based on the drone's target turning radius, drone motion step length, and target heading angle; Construct the target neural network; The exploration map explored by the drone in an unknown environment is input into the first convolutional layer to obtain the feature map of the exploration map; Input the feature map of the exploration map into several transform modules to obtain the feature vector of the exploration map; The feature vector of the exploration map is input into the second convolutional layer to obtain the predicted map corresponding to the initial digital elevation map.

2. The method according to claim 1, characterized in that The initial digital elevation map is normalized to obtain the target digital elevation map expression: in, is the pixel value of each point in the target digital elevation map, is the pixel value of each point in the initial digital elevation map, is the pixel value of the lowest point in the initial digital elevation map, It is the pixel value of the highest point in the initial digital elevation map.

3. The method according to claim 2, characterized in that The method of dividing the target digital elevation map by using the threshold height to obtain a planar two-dimensional map corresponding to the target digital elevation map includes: When the pixel value corresponding to the pixel point in the target digital elevation map is greater than the threshold height, the pixel value of the pixel point in the target digital elevation map is set to one; When the pixel value corresponding to the pixel point in the target digital elevation map is less than the threshold height, the pixel value of the pixel point in the target digital elevation map is set to zero; Based on the pixel values ​​of zero and one, a flat two-dimensional map is obtained.

4. The method according to claim 3, characterized in that The method of determining the initial flyable point of the UAV based on the planar two-dimensional map and the initial flight point includes: Determine the candidate flight points of the UAV based on the planar 2D map and the initial flight points; Any point of the candidate flight point of the UAV is used as the initial flyable point of the UAV.

5. The method according to claim 4, characterized in that The method of determining the target turning radius, the movement step length and the target heading angle of the UAV on the two-dimensional map based on the target speed, the gravity acceleration, the maximum allowable normal overload factor and the target time interval of the UAV at the initial flyable point of the UAV includes: Determine the target turning radius based on the target speed of the UAV at the initial flyable point, the acceleration of gravity, and the maximum allowable normal overload factor of the UAV; Determine the UAV motion step length based on the UAV's target speed and target time interval at the initial flyable point; The target heading angle is determined based on the UAV motion step length and the target turning radius.

6. The method according to claim 5, characterized in that The expression of the target turning radius is: Where g is the acceleration due to gravity, is the maximum allowable normal overload factor of the UAV, is the target speed of the UAV, is the target turning radius.

7. The method according to claim 6, characterized in that The expression of the UAV motion step length is: in, is the motion step length of the drone, is the target time interval.

8. The method according to claim 7, characterized in that The expression of the target heading angle is: in, is the target heading angle.

9. The method according to claim 8, characterized in that The method of determining the exploration map explored by the drone in an unknown environment based on the target turning radius of the drone, the motion step length of the drone, and the target heading angle includes: Based on the target turning radius of the UAV, the UAV movement step length and the target heading angle, the position, navigation direction and yaw angle of the UAV in the flat two-dimensional map at each moment are determined; Based on the position, navigation direction and yaw angle in the flat two-dimensional map at each moment, the exploration map explored by the drone in an unknown environment is determined.

10. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 1.

Images