Unknown environment safety navigation method and system based on dynamic weighting of front risk

Abstract

The application discloses a method and system for safe navigation in unknown environment based on dynamic weighting of front risk, which comprises the following steps: S1: obtaining laser radar observation data of the current time of the unmanned aerial vehicle; S2: obtaining the current motion instruction of the unmanned aerial vehicle; S3: constructing and calculating the continuous front risk value at the current time; S4: performing time recursive smoothing processing on the continuous front risk value to obtain the memory risk value at the current time; S5: dividing to obtain the discrete front risk level at the current time; S6: dynamically matching to obtain the dynamic penalty weight at the current time; S7: constructing the adaptive front risk penalty term; and S8: outputting the current action instruction for controlling the flight of the unmanned aerial vehicle. The application balances the exploration efficiency and safety constraints in real time in the reinforcement learning navigation by continuously quantifying the front risk, establishing the memory and grading mechanism and adaptively constructing the dynamic penalty term, and improves the navigation safety and robustness of the unmanned aerial vehicle in the unknown environment.

Description

Technical Field

[0001] This invention relates to the fields of autonomous navigation and artificial intelligence technology for unmanned aerial vehicles (UAVs), specifically to a method and system for safe navigation in unknown environments based on dynamic weighting of cutting-edge risks. Background Technology

[0002] Unmanned aerial vehicles (UAVs) have been widely used in disaster search and rescue, emergency reconnaissance, underground space inspection, indoor and outdoor 3D mapping, and autonomous flight in complex environments due to their advantages such as high mobility, flexible deployment, and ability to enter dangerous or complex areas. In these scenarios, UAVs typically need to complete autonomous navigation, environmental exploration, and safe obstacle avoidance under conditions of lack of prior maps, unknown obstacle distribution, and limited perception range. Therefore, autonomous navigation in unknown environments has become an important research direction for the intelligent development of UAVs.

[0003] From a technical perspective, navigation in unknown environments typically involves multiple stages, including environmental perception, pose estimation, map building, frontier search, path planning, trajectory generation, and flight control. Unmanned aerial vehicles (UAVs) need to utilize LiDAR, depth cameras, visual sensors, and inertial measurement units to acquire environmental information, build local or global maps based on this information, and then determine the next flight direction according to the distribution of free space, obstacle space, and unknown space. Because UAVs operate in three-dimensional space and are limited by onboard computing power, endurance, and sensor field of view, the system must not only meet real-time requirements but also balance navigation safety, trajectory feasibility, and exploration efficiency.

[0004] Currently, drone navigation in unknown environments is mainly developing along two technical routes: The first category is modular methods based on environmental mapping and path planning. These methods typically rely on depth cameras, LiDAR, or visual-inertial navigation systems to acquire environmental information, construct occupancy grid maps, voxel maps, or range field maps, and extract leading-edge regions at the boundary between known and unknown spaces. These regions serve as subsequent exploration targets. Candidate viewpoints are then selected by combining indicators such as path cost, information gain, visibility, and reachability. Finally, graph search, sampling planning, or trajectory optimization algorithms are used to generate flight paths. While these methods have a clear structure, mature engineering implementation, and are easy to integrate with existing flight control systems, their focus is usually on coverage, path length, information gain, and trajectory executability, lacking specific characterization of potential risks near the boundaries of unknown regions.

[0005] The second category is autonomous decision-making methods based on deep reinforcement learning. These methods directly input sensory information and flight status into the policy network, outputting speed, heading angle, or attitude control variables. A reward function comprehensively considers indicators such as target arrival, collision avoidance, path efficiency, and energy consumption. Some schemes also introduce curiosity mechanisms, safety shielding, or action correction modules to improve exploration capabilities and flight safety. While these schemes have certain advantages in environmental adaptability and autonomous learning capabilities, their reward designs mostly revolve around collision, obstacle distance, and target arrival, and have not yet established continuous risk representation, risk level classification, and dynamic penalty weight adjustment mechanisms for the frontier regions in unknown environments.

[0006] Specifically, existing technical solutions mainly include cutting-edge-driven autonomous exploration solutions and reinforcement learning-based safe navigation solutions.

[0007] Frontier-driven autonomous exploration schemes typically begin by acquiring environmental information using airborne depth cameras, lidar, or visual sensors, and then constructing local or global maps based on pose estimation results. Next, frontier boundaries are extracted at the boundary between free and unknown spaces, with the frontier region serving as a candidate exploration target. Then, candidate viewpoints are generated around the frontier region and evaluated based on factors such as path cost, information gain, visibility, connectivity, and reachability to determine the next exploration direction. Subsequently, executable paths from the current position to the target viewpoint are generated using methods such as A / R, RRT, or trajectory optimization, and tracked by the flight control system. This type of scheme has a clear structure, mature engineering implementation, and is easy to integrate with existing flight control systems. However, it usually focuses more on the information gains of the frontier region than on the inherent uncertainties and risks of the frontier region itself, thus lacking continuous quantification and dynamic mitigation mechanisms for frontier risks.

[0008] Reinforcement learning-based safe navigation schemes typically model the UAV navigation process as a Markov decision process, using depth images, LiDAR distance information, target relative direction, and velocity and attitude as state inputs. Control commands are directly output through a policy network, and a reward function is used to jointly optimize target arrival, collision penalties, obstacle distance, path efficiency, and energy consumption. Some studies also introduce intrinsic rewards or curiosity mechanisms to enhance the UAV's proactive exploration capabilities in unknown areas; others introduce shield modules, safety layers, or action verification mechanisms to modify policy output actions and reduce collision risks. While these schemes have certain advantages in environmental adaptability and autonomous learning capabilities, their reward designs mostly revolve around collision, obstacle distance, and target arrival, and have not yet established continuous risk representation, risk level classification, and dynamic penalty weight adjustment mechanisms for the frontier areas in unknown environments. Therefore, in complex and unknown environments, it remains difficult to balance exploration gains and safety constraints in real time.

[0009] In summary, the existing technology has the following main drawbacks: First, there is a lack of specific characterization of risks in frontier areas. Existing frontier-driven exploration methods typically use path cost, information gain, visibility, and accessibility as criteria for target selection, focusing on improving exploration efficiency, but lack a specific description of the uncertainty risks brought about by the unknown space adjacent to the frontier area.

[0010] Second, it is difficult to continuously quantify potential risks near unknown boundaries. Existing methods can usually identify the risks of observed obstacles, but they lack a continuous quantification mechanism for potential risks that may exist when approaching the boundaries of unknown regions, making it difficult to distinguish the risk differences under different frontier locations and different approach methods.

[0011] Third, there is a lack of dynamic constraint mechanisms to address frontier risks. Most penalty terms or safety constraints in existing safe navigation methods are fixed in form and cannot dynamically adjust the constraint strength according to changes in frontier area risks. Therefore, they are difficult to adapt to the real-time navigation needs in complex and unknown environments.

[0012] Although the above methods can achieve autonomous navigation in unknown environments to a certain extent, existing technologies still struggle to simultaneously balance exploration efficiency, edge-following risk mitigation, and navigation robustness. Summary of the Invention

[0013] To overcome the shortcomings of existing technologies, one of the objectives of this invention is to provide a safe navigation method for unknown environments based on dynamic weighting of frontier risks. This method continuously quantifies frontier risks, establishes memory and hierarchical mechanisms, and adaptively constructs dynamic penalty terms to balance exploration efficiency and safety constraints in real time during reinforcement learning navigation, thereby improving the navigation safety and robustness of UAVs in unknown environments.

[0014] The second objective of this invention is to provide a safe navigation system for unknown environments based on dynamic weighting of frontier risks. This system continuously quantifies frontier risks, establishes memory and hierarchical mechanisms, and adaptively constructs dynamic penalty terms to balance exploration efficiency and safety constraints in real time during reinforcement learning navigation, thereby improving the navigation safety and robustness of UAVs in unknown environments.

[0015] To achieve one of the objectives of this invention, the following solution is adopted: A method for safe navigation in unknown environments based on dynamic weighting of cutting-edge risks includes the following steps: S1: Obtain the lidar observation data of the UAV at the current moment, and calculate the proportion of unknown cue beams in multiple different directions based on the lidar observation data to obtain the unknown proportion in each direction; S2: Obtain the current motion command of the UAV, and calculate the effective motion components in the multiple different directions based on the current motion command; S3: Based on the unknown proportions in each direction and the effective motion components, construct and calculate the continuous frontal risk value at the current moment to quantify the potential risk of the UAV advancing toward the unknown area; S4: Perform time-recursive smoothing on the continuous frontier risk values ​​to obtain the memory risk value at the current moment; S5: Based on the comparison result between the memory risk value and at least one preset risk threshold, the discrete frontier risk level at the current moment is determined; S6: Based on the discrete frontier risk level, the dynamic penalty weight at the current moment is dynamically matched and obtained through the preset mapping relationship between risk level and penalty weight; S7: Based on the dynamic penalty weight and the memory risk value, construct an adaptive frontier risk penalty term, and use the adaptive frontier risk penalty term to comprehensively evaluate at least one candidate local navigation target, and select the local navigation target at the current time from the at least one candidate local navigation target; S8: Input the current state, which includes the continuous front risk value, the memory risk value, the discrete front risk level, the local navigation target, and the current environmental perception information and motion state information of the UAV, into the pre-trained reinforcement learning policy network to output the current action command for controlling the flight of the UAV.

[0016] Further, in step S1, the proportion of unknown cue beams in multiple different directions is calculated based on the lidar observation data to obtain the unknown proportion in each direction, specifically including: The multiple laser beams in the lidar observation data are divided into forward, left, and right regions according to the body coordinate system. For each region, the number of laser beams with a measurement distance close to the maximum range of the lidar is counted, and the ratio of this number to the total number of laser beams in the region is taken as the proportion of unknown direction for the corresponding region.

[0017] Further, in step S3, based on the unknown proportions in each direction and the effective motion components, the continuous frontier risk value at the current moment is constructed and calculated, specifically including: The weighted sum of the unknown proportions of the forward, left, and right directions and the effective motion components is used as the continuous frontier risk value.

[0018] Further, in step S4, the step of performing time-recursive smoothing on the continuous frontier risk values ​​to obtain the memory risk value at the current moment specifically includes: Obtain the memory risk value from the previous moment; Based on a preset risk decay coefficient, an exponentially weighted moving average is calculated on the memory risk value of the previous moment and the continuous frontier risk value of the current moment, and the result is used as the memory risk value of the current moment.

[0019] Further, in step S5, the step of determining the discrete frontier risk level at the current moment based on the comparison result between the memory risk value and at least one preset risk threshold specifically includes: When the memory risk value is less than or equal to the first preset threshold, it is determined to be a low risk level; When the memory risk value is greater than the first preset threshold and less than or equal to the second preset threshold, it is determined to be at a medium risk level. When the memory risk value is greater than the second preset threshold, it is determined to be a high-risk level.

[0020] Further, in step S6, the step of dynamically matching and obtaining the dynamic penalty weight at the current moment based on the discrete frontier risk level and through a preset mapping relationship between risk level and penalty weight specifically includes: A first penalty weight, a second penalty weight, and a third penalty weight are preset to correspond one-to-one with low-risk level, medium-risk level, and high-risk level, wherein the first penalty weight, the second penalty weight, and the third penalty weight increase sequentially; When the discrete frontier risk level is low, the first penalty weight is matched as the dynamic penalty weight; when the discrete frontier risk level is medium, the second penalty weight is matched as the dynamic penalty weight; when the discrete frontier risk level is high, the third penalty weight is matched as the dynamic penalty weight. In step S7, the adaptive frontier risk penalty term is obtained by calculating the product of the dynamic penalty weight and the memory risk value.

[0021] Further, in step S7, the step of comprehensively evaluating at least one candidate local navigation target using the adaptive frontier risk penalty term and selecting the local navigation target at the current moment from the at least one candidate local navigation target specifically includes: Obtain the basic benefit evaluation and unknown propulsion risk cost for each candidate target among at least one candidate local navigation target; For each candidate target, the comprehensive evaluation function value of the candidate target is obtained by subtracting the product of the adaptive frontier risk penalty term and the unknown propulsion risk cost from the basic benefit evaluation. The candidate target with the largest comprehensive evaluation function value is selected as the local navigation target at the current moment.

[0022] Furthermore, in step S8, the input state of the pre-trained reinforcement learning policy network also includes the relative position or direction information of the local navigation target at the current moment; the current action command includes one or more of the UAV's desired speed, heading adjustment amount, or attitude correction amount.

[0023] Furthermore, in step S8, the pre-trained reinforcement learning policy network adopts a proximal policy optimization algorithm and is trained using an immediate reward function; the immediate reward function includes a task advancement reward, a basic safety constraint reward, and an adaptive frontier risk penalty.

[0024] To achieve the second objective of this invention, the following solution is adopted: A safety navigation system for unknown environments based on dynamic weighted assessment of cutting-edge risks includes: The unknown proportion calculation module is used to acquire the lidar observation data of the UAV at the current moment, and calculate the proportion of unknown cue beams in multiple different directions based on the lidar observation data to obtain the unknown proportion in each direction. An effective motion component calculation module is used to obtain the current motion command of the UAV and calculate the effective motion components in multiple different directions based on the current motion command; The continuous front risk construction module is used to construct and calculate the continuous front risk value at the current moment based on the unknown proportions in each direction and the effective motion components, so as to quantify the potential risk of the UAV advancing toward the unknown area. The memory risk smoothing module is used to perform time-based smoothing on the continuous leading edge risk values ​​to obtain the memory risk value at the current moment. The risk level classification module is used to classify the discrete frontier risk level at the current moment based on the comparison result between the memory risk value and at least one preset risk threshold. The dynamic penalty weight matching module is used to dynamically match the dynamic penalty weight at the current moment based on the discrete frontier risk level and through a preset mapping relationship between risk level and penalty weight. The local navigation target selection module is used to construct an adaptive frontier risk penalty term based on the dynamic penalty weight and the memory risk value, and to use the adaptive frontier risk penalty term to comprehensively evaluate at least one candidate local navigation target, and select the local navigation target at the current time from the at least one candidate local navigation target; The action command output module is used to input the current state, which includes the continuous front risk value, the memory risk value, the discrete front risk level, the local navigation target, and the current environmental perception information and motion state information of the UAV, into a pre-trained reinforcement learning policy network, so as to output the current action command for controlling the flight of the UAV.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention achieves continuous quantification and dynamic perception of risks in the forward region. By acquiring lidar observation data to calculate the proportion of unknowns in multiple directions and combining it with the effective motion components of the current motion command to construct a continuous forward risk value, it is possible to quantify the potential risks faced by UAVs when advancing towards unknown areas in real time and continuously, overcoming the shortcomings of existing technologies that are unable to distinguish the differences in risks under different forward positions and different approach methods.

[0026] 2. This invention establishes a risk smoothing and grading mechanism based on the memory effect. By performing time-recursive smoothing on continuous frontier risk values, a memory risk value is obtained, avoiding misjudgments caused by single-observation noise. Then, based on the comparison between the memory risk value and a preset risk threshold, discrete frontier risk levels are classified, achieving stable identification and grading of risk states, providing a reliable basis for subsequent dynamic adjustments.

[0027] 3. This invention achieves dynamic penalty weight adaptive matching for frontier risks. Based on the defined discrete frontier risk levels, the dynamic penalty weight at the current moment is dynamically matched through a preset mapping relationship between risk levels and penalty weights. This allows the penalty intensity to adaptively adjust with the real-time risk level, solving the problem in existing technologies where penalty items or safety constraints are fixed and difficult to adapt to the real-time navigation needs of complex and unknown environments.

[0028] 4. This invention constructs an adaptive frontier risk penalty term, optimizing the selection of local navigation targets. Based on dynamic penalty weights and memorized risk values, an adaptive frontier risk penalty term is constructed and used for the comprehensive evaluation of candidate local navigation targets. This proactively suppresses directional behavior in high-risk frontier regions during the exploration process, effectively reducing risk while ensuring exploration efficiency, and achieving a dynamic balance between exploration gains and safety constraints.

[0029] 5. This invention improves the safety and robustness of reinforcement learning navigation strategies. By inputting continuous frontier risk values, memorized risk values, discrete frontier risk levels, and selected local navigation targets, along with the UAV's current environmental perception and motion state information, into a pre-trained reinforcement learning strategy network, the network can perceive the frontier risk state and output safer and more reasonable flight control commands, significantly improving the navigation safety and robustness of UAVs in unknown and complex environments. Attached Figure Description

[0030] Figure 1 This is a flowchart of the unknown environment safety navigation method based on dynamic weighting of frontier risks in an embodiment of the present invention; Figure 2This is a flowchart of the UAV forward flight space risk classification method in an embodiment of the present invention; Figure 3 This is a flowchart of the local navigation target dynamic decision-making method based on risk level in an embodiment of the present invention; Figure 4 This is a schematic diagram of the unknown environment safety navigation method based on dynamic weighting of frontier risks in an embodiment of the present invention; Figure 5 This is a block diagram of a safe navigation system for unknown environments based on dynamic weighting of frontier risks, as described in an embodiment of the present invention. Detailed Implementation

[0031] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0032] Terminology Explanation: Continuous Frontier Risk Value: A continuous risk index calculated based on the current direction of the drone's movement and the proportion of unknown areas in each direction. It is used to quantify the potential safety risk intensity of the current exploration behavior entering unknown areas.

[0033] Memory risk value: The risk level obtained by time-based recursion and smoothing of continuous risk values. It not only considers the current risk but also retains historical risk trends, reducing the impact of sensor noise or instantaneous fluctuations and making risk assessment more stable.

[0034] Frontier Risk Level: A discrete risk category obtained by dividing the memory risk value through a preset threshold, used to describe the safety status of the current navigation behavior.

[0035] Dynamic penalty weight: A penalty coefficient of the reward function that is adaptively adjusted according to the risk level or navigation status, used to regulate the trade-off between exploration behavior and safety constraints.

[0036] Example 1 like Figure 1-4 As shown, this embodiment of the invention provides a safe navigation method for unknown environments based on dynamic weighting of frontier risks, including the following steps: S1: Obtain the lidar observation data of the UAV at the current moment, and calculate the proportion of unknown cue beams in multiple different directions based on the lidar observation data to obtain the unknown proportion in each direction.

[0037] S2: Obtain the current motion command of the UAV, and calculate the effective motion components in the multiple different directions based on the current motion command.

[0038] S3: Based on the unknown proportions in each direction and the effective motion components, construct and calculate the continuous frontal risk value at the current moment to quantify the potential risk of the UAV advancing toward the unknown area.

[0039] S4: Perform time-recursive smoothing on the continuous frontier risk values ​​to obtain the memory risk value at the current moment.

[0040] S5: Based on the comparison result between the memory risk value and at least one preset risk threshold, the discrete frontier risk level at the current moment is determined.

[0041] S6: Based on the discrete frontier risk level, the dynamic penalty weight at the current moment is dynamically matched and obtained through the preset mapping relationship between risk level and penalty weight.

[0042] S7: Based on the dynamic penalty weight and the memory risk value, construct an adaptive frontier risk penalty term, and use the adaptive frontier risk penalty term to comprehensively evaluate at least one candidate local navigation target, and select the local navigation target at the current time from the at least one candidate local navigation target.

[0043] S8: Input the current state, which includes the continuous frontier risk value, the memory risk value, the discrete frontier risk level, the local navigation target, and the current environmental perception information and motion state information of the UAV, into the pre-trained reinforcement learning policy network to output the current action command for controlling the flight of the UAV.

[0044] The unknown environment safety navigation method based on dynamic weighted frontier risk in this invention continuously quantifies frontier-related risks and further classifies and dynamically adjusts risk levels. This allows the navigation strategy to adaptively adjust constraint strength according to changes in frontier region risks, thereby more reasonably balancing exploration efficiency and flight safety during unknown environment exploration. This method addresses the problems in existing technologies, such as the lack of specific characterization of frontier risks, insufficient continuity of risk constraints, and difficulty in dynamically changing penalty mechanisms. It enables UAVs not only to achieve autonomous exploration in unknown environments but also to effectively suppress high-risk flight behaviors close to unknown boundaries, improving navigation safety, stability, and robustness in complex environments while ensuring safety and exploration efficiency.

[0045] The following is a more detailed description of the safe navigation method for unknown environments based on dynamic weighting of frontier risks according to embodiments of the present invention.

[0046] The unknown environment safety navigation method based on dynamic weighting of frontier risks in this invention is a method for safe navigation of UAVs in unknown environments that integrates a UAV forward flight space risk classification method with a navigation target dynamic weight adjustment mechanism. It addresses the problems of UAVs' inability to effectively characterize the forward flight space risks during unknown environment exploration and the lack of adaptive adjustment of navigation decisions with changes in environmental risks. By establishing a UAV forward flight space risk classification method and a navigation target dynamic weight adjustment mechanism, the UAV can adjust its navigation behavior in real time according to changes in local environmental risks, thereby improving navigation safety and decision robustness while ensuring exploration efficiency.

[0047] In this embodiment, the specific content of the UAV forward flight space risk classification method is as follows: In autonomous navigation missions in unknown environments, the local environment can typically be divided into known free regions, obstacle-occupied regions, and unknown regions. Free regions represent spaces observed by onboard sensors and safe for the UAV to navigate; obstacle-occupied regions represent spaces identified as impassable; and unknown regions represent spaces that have not yet been observed or whose status has not been confirmed. As the UAV continues to fly and acquire local observation information, the known free regions gradually expand into the unknown regions, forming a leading edge region at their boundary.

[0048] If a free raster or free voxel in a local map is adjacent to at least one unknown raster or unknown voxel, then that free raster or unknown voxel is defined as a frontier cell. The extracted frontier cells are then clustered to form frontier clusters with spatial continuity, which help characterize the distribution of unknown regions in local space. Existing frontier detection methods can be used to extract the frontier cells or frontier clusters.

[0049] After obtaining local spatial observation information, the airborne lidar scan results are used to identify unknown areas and indicate directions. Let the lidar currently obtain a total of... The first laser beam The measurement distance of the laser beam is denoted as The maximum range of the lidar is recorded as The range determination margin is recorded as Furthermore, to characterize the distribution characteristics of unknown areas in different directions of the UAV, the entire laser beam is divided into three fan-shaped sub-regions according to the aircraft coordinate system: the forward region, the left region, and the right region, denoted as […]. , and For any region (in The proportion of its unknown cue bundle is defined as: Among them, when the first Measurement distance of a single laser beam Approaching the maximum range At that time, it indicates that there is an unknown area in that direction, and a warning message is displayed. This is the indicator function. From this, the proportion of the unknown forward direction can be obtained. Unknown percentage of left-hand side and the unknown proportion to the right It is used to quantify the distribution of unknown areas in three directions in the current local observation of the UAV.

[0050] Based on this, to characterize the coupling relationship between the current action command of the UAV and the distribution of unknown regions in different directions, the effective motion components in the forward, left, and right directions are defined as follows: in, Forward linear velocity command, This refers to the lateral linear velocity command. The definition indicates that a risk contribution only occurs in the corresponding direction if the UAV's current action is actually moving in that direction.

[0051] Based on the aforementioned proportion of directional unknown cue beams With directional motion components Continuous risk value Defined as: in, This is a very small positive number used to ensure numerical stability. Therefore, it can be seen that when the drone moves towards a direction with a higher proportion of unknown areas, the continuous risk value increases; when the drone mainly moves towards known safe areas or does not move towards directions with high concentrations of unknown areas, the continuous risk value decreases.

[0052] Considering the suddenness and short-term persistence of risks in unknown areas, and to avoid frequent oscillations in risk values ​​caused by fluctuations observed at a single moment, a risk memory mechanism is further introduced, defining a memory risk value. for: in, This is the risk attenuation coefficient. When the current risk increases, the memory risk value can respond quickly; when the current risk decreases, the memory risk value is released gradually in an attenuation manner to improve the stability of risk assessment.

[0053] Furthermore, based on the smoothed risk value The current environmental risks are classified into multiple levels: in, and Preset risk thresholds are used to represent low, medium, and high risk states, respectively. Based on these risk levels, the drone's speed limit, maneuver strategy, or local planning method can be further adjusted to improve flight safety in unknown environments.

[0054] In this embodiment, the specific details of the local navigation target dynamic decision-making method based on risk level are as follows: The continuous risk value obtained based on the aforementioned steps Memory risk value and discrete risk level To enable UAVs to adaptively adjust their propulsion strategies based on the current risk level in unknown environments, this embodiment further proposes a local navigation target dynamic decision-making method based on risk levels. This method maps discrete risk levels to risk penalty weights of varying intensities, dynamically adjusting the degree to which the navigation strategy suppresses high-risk propulsion behaviors, thereby achieving a balance between navigation efficiency and flight safety.

[0055] First, a set of penalty weights corresponding one-to-one with discrete risk levels is predefined: in, , and These correspond to penalty weight coefficients for low-risk, medium-risk, and high-risk states, respectively. To ensure the adjustment logic that higher risk levels result in greater penalty intensity, the penalty weight set satisfies the following monotonically increasing constraint: In the Each navigation decision step, based on the aforementioned risk assessment steps, yields the discrete risk level at the current moment. And through the risk level mapping function Dynamic matching of corresponding penalty weight coefficients ,Right now: in, .

[0056] The penalty weight in this embodiment is not directly determined based on general obstacle density or ordinary environmental cost, but is dynamically mapped based on the discrete risk level obtained by combining the aforementioned unknown cue laser beam, the proportion of unknown direction, directional motion components and risk memory mechanism. Therefore, it can more accurately reflect the actual risk status of the UAV when it is advancing towards an unknown area.

[0057] The dynamically generated penalty weight coefficient With current memory risk value Combined, construct an adaptive frontier risk penalty term: in, Indicates the first Each decision step involves a risk penalty. The penalty weight increases with the risk level. The risk penalty term is gradually increased, thereby increasing the absolute value of the risk penalty term in tandem, which enhances the strategy's tendency to avoid high-risk, unknown areas.

[0058] Under different risk levels, the above dynamic weight adjustment mechanism has the following effects: 1. When When this occurs, it indicates that the current environment is in a low-risk state. At this time, the system assigns a smaller penalty weight. This reduces the inhibitory effect of risk penalties on propulsion, allowing the drone to prioritize maintaining its flight trend toward the target area, thereby improving navigation efficiency.

[0059] 2. When When this occurs, it indicates that the current environment is in a medium-risk state. At this time, the system assigns a medium penalty weight. While maintaining the ability to advance towards the target, the cost of high-risk maneuvers is appropriately increased, prompting the drone to adjust its course, speed, or attitude in order to balance exploration efficiency and passage safety.

[0060] 3. When When this occurs, it indicates that the current environment is in a high-risk state. At this time, the system assigns a larger penalty weight. This significantly increases the cost of high-risk maneuvers and guides drones to prioritize more conservative and safe maneuvers such as deceleration, detour, hovering, or withdrawal, in order to reduce the probability of collisions or accidental entry into high-risk unknown areas.

[0061] Specifically, the dynamic penalty weight It can be used for the comprehensive evaluation of candidate local navigation targets. Let the set of candidate local navigation targets at the current time be: For any candidate target Construct its comprehensive evaluation function: in, The basic benefit evaluation of candidate targets may be composed of one or more of the following: target proximity, reachability, and degree of support in the free region. This represents the unknown propulsion risk cost corresponding to the candidate target. Ultimately, the candidate target with the highest comprehensive evaluation value is selected as the local navigation target for the current moment. Therefore, when the environmental risk is low, the system tends to select local targets that are more conducive to mission advancement; when the environmental risk is high, the system increases the penalty weight to raise the risk cost of high-risk candidate targets, thereby suppressing the selection of candidate targets that advance towards high-risk unknown areas, thus achieving dynamic adaptive decision-making for local navigation targets.

[0062] After completing the aforementioned continuous risk calculation, memory risk update, risk level classification, and local navigation target dynamic decision-making, this embodiment further employs a near-end strategy optimization algorithm to train the UAV's local navigation strategy, so that the constructed risk assessment results and dynamic target decision-making mechanism can be effectively utilized during the strategy learning process.

[0063] In the Each decision step constructs the policy input state for the drone. The state This includes at least current environmental perception information, UAV motion status information, target relative pose information, and the aforementioned risk assessment results. Furthermore, in some embodiments, the state... It also includes the local navigation target selected at the current moment. The relative position or orientation information allows the policy network to combine the current security risk state and the local target distribution to output control actions. The state can be represented as: in, This indicates current environmental observation information. This indicates the drone's own motion status information. Represents the relative pose information of the global target or mission target. This represents the local navigation target obtained through dynamic decision-making at the current moment. Indicates continuous risk value, Indicates the memory risk value. This indicates the discrete risk level.

[0064] The policy network is in the state described. As input, the current action command is output. : in, The parameter is The strategy network. The action instructions. It may include one or more of the following: the desired speed of the UAV, the heading adjustment, and the attitude correction.

[0065] After the action is performed, the environment returns the corresponding immediate reward. and the state at the next moment The immediate reward function can be expressed as: in, This indicates the rewards for completing the task. This indicates the basic security constraint reward items. This represents a risk penalty item dynamically constructed based on the current risk level. Through this reward design, the policy network not only learns to advance towards the target area during training, but also adaptively adjusts its advancement intensity and risk-avoidance behavior according to the current unknown environmental risk level.

[0066] To limit the magnitude of a single policy update and improve training stability, this implementation uses the shearing objective function from the PPO algorithm to update the policy network parameters. Its policy optimization objective can be expressed as: in, This indicates the old strategy before the update. This represents the estimated value of the advantage function. This represents the policy truncation coefficient. Through this truncation mechanism, excessive changes in policy parameters during a single iteration can be avoided, thereby improving the convergence stability of the training process.

[0067] The PPO training process can also incorporate the value function loss term and the policy entropy regularization term to construct the overall optimization objective, balancing the accuracy of state value estimation and policy exploration capability. After training, the converged policy network is deployed to the UAV online navigation system. During the online execution phase, only the current environmental perception information, motion state information, local target information, and risk assessment results need to be input into the policy network to output corresponding control actions in real time, thereby achieving safe local navigation in unknown environments.

[0068] In summary, the UAV local navigation method in unknown environments provided in this embodiment includes the following steps: First, extract unknown cue laser beam information based on lidar observation results, and construct a continuous risk value by combining the proportion of unknown directions and the motion direction component; second, smoothly update the continuous risk value through a time-recursive method to obtain a memory risk value, and classify it into discrete risk levels according to a preset threshold; then, dynamically map risk penalty weights according to the discrete risk levels to construct an adaptive risk penalty term, and comprehensively evaluate candidate local navigation targets accordingly to select the local navigation target at the current moment; finally, input the environmental state, target information, and risk assessment results into the PPO policy network, output the UAV control actions, and realize active avoidance of unknown high-risk areas and stable advancement of target areas.

[0069] This invention presents a risk assessment method based on the coupling of unknown region distribution and motion direction. It extracts relevant information about unknown regions from lidar observations and combines this information with the proportion of unknowns in the current navigation direction and the actual motion direction of the UAV to construct a continuous risk value, which characterizes the potential risk of "advancing towards an unknown region." This risk definition method is one of the key innovations of this invention.

[0070] This invention employs a risk memory and discrete grading mechanism that considers temporal continuity. Continuous risk values ​​are updated over time through recursion and smoothing to obtain memory risk values, which are then classified into different risk levels based on preset thresholds. This mechanism can suppress decision-making oscillations caused by instantaneous noise and local fluctuations, improving the stability and reliability of risk assessment.

[0071] This invention presents a dynamic decision-making method for local navigation targets based on risk levels. The risk level is mapped to a dynamic penalty weight, which is used for the comprehensive evaluation of candidate local navigation targets or the adjustment of strategy rewards. This allows the system to prioritize task progress efficiency when the risk is low, and to prioritize suppressing the selection of targets that advance into high-risk unknown areas when the risk is high, thereby achieving an adaptive balance between safety and navigation efficiency.

[0072] The safe navigation method for unknown environments based on dynamic weighted frontier risk according to the embodiments of the present invention has the following advantages: The unknown environment safety navigation method based on dynamic weighted frontier risk in this invention no longer relies solely on path cost, information gain, or the distance to observed obstacles for navigation decisions. Instead, it further establishes a risk characterization mechanism for the frontier region in the unknown environment, continuously quantifies uncertainties related to the frontier, and adaptively adjusts the constraint strength or penalty weight during navigation by combining risk level classification and dynamic adjustment strategies. This enables the UAV to consider not only "whether to continue exploration" when approaching the boundary of an unknown region, but also "how great the potential risk is in this direction," thereby improving the pertinence and rationality of decision-making. The unknown environment safety navigation method based on dynamic weighted frontier risk in this invention can more effectively suppress high-risk flight behaviors close to unknown boundaries, reducing the problem of passive avoidance only after a collision occurs or an obstacle is fully observed, giving the UAV better forward-looking safety control capabilities in complex unknown environments. Compared with existing technologies, this invention, while ensuring exploration capabilities, is more conducive to balancing exploration efficiency and flight safety, thereby improving the stability, robustness, and environmental adaptability of the UAV during autonomous navigation.

[0073] In other embodiments, the quantitative calculation of continuous risk values ​​at the forefront can employ probabilistic modeling methods or virtual force field methods based on uncertainty estimation instead of geometric projection statistics. For example, Bayesian neural networks or Gaussian process regression can be used to model the environment, calculate the information entropy or variance of the leading edge region, and directly define high-uncertainty regions as high-risk sources; or a variant of the artificial potential field method can be used, treating unknown regions in the direction of action as obstacles that generate "virtual repulsive forces," and characterizing the continuous risk coupled with the motion intention by calculating the magnitude of the resultant repulsive force. These methods also achieve the goal of "quantifying environmental risk based on motion intention."

[0074] In other embodiments, the risk grading mechanism for suppressing perceived noise can replace the multi-level fixed threshold discretization method with a combination of nonlinear continuous mapping and time-domain filtering or hysteresis comparison mechanism. For example, the original continuous risk value can be nonlinearly mapped to a normalized interval using Sigmoid, Tanh, or exponential functions, and then smoothed using moving average filtering or Kalman filtering algorithms; or a hysteresis comparator (dual threshold) logic can be used to set different threshold lines for increasing and decreasing the risk level to avoid frequent jumps in values ​​near the critical point. These methods can also achieve the effect of filtering out high-frequency observation noise and stabilizing the control strategy.

[0075] In other embodiments, risk-level-based decision adjustment mechanisms can replace modifying the reward function (adding adaptive penalty weights) with action masking or constrained optimization frameworks. For example, based on the current risk level, action options pointing to high-risk regions can be directly and forcibly masked at the output layer of the reinforcement learning network, reducing their selection probability to zero; or a constrained Markov decision process framework can be used, treating risk as an independent cost signal rather than part of the reward, and dynamically adjusting the policy's tolerance for risk costs using the Lagrange multiplier method. These methods can also achieve the goal of "dynamically balancing exploration and safety according to the degree of risk."

[0076] Example 2 like Figure 5 As shown, embodiments of the present invention also provide a safety navigation system for unknown environments based on dynamic weighting of frontier risks, including: The unknown proportion calculation module is used to acquire the lidar observation data of the UAV at the current moment, and calculate the proportion of unknown cue beams in multiple different directions based on the lidar observation data to obtain the unknown proportion in each direction; An effective motion component calculation module is used to obtain the current motion command of the UAV and calculate the effective motion components in multiple different directions based on the current motion command; The continuous front risk construction module is used to construct and calculate the continuous front risk value at the current moment based on the unknown proportions in each direction and the effective motion components, so as to quantify the potential risk of the UAV advancing toward the unknown area. The memory risk smoothing module is used to perform time-based smoothing on the continuous leading edge risk values ​​to obtain the memory risk value at the current moment. The risk level classification module is used to classify the discrete frontier risk level at the current moment based on the comparison result between the memory risk value and at least one preset risk threshold. The dynamic penalty weight matching module is used to dynamically match the dynamic penalty weight at the current moment based on the discrete frontier risk level and through a preset mapping relationship between risk level and penalty weight. The local navigation target selection module is used to construct an adaptive frontier risk penalty term based on the dynamic penalty weight and the memory risk value, and to use the adaptive frontier risk penalty term to comprehensively evaluate at least one candidate local navigation target, and select the local navigation target at the current time from the at least one candidate local navigation target; The action command output module is used to input the current state, which includes the continuous front risk value, the memory risk value, the discrete front risk level, the local navigation target, and the current environmental perception information and motion state information of the UAV, into a pre-trained reinforcement learning policy network, so as to output the current action command for controlling the flight of the UAV.

[0077] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for safe navigation in unknown environments based on dynamic weighting of frontier risks, characterized in that, Includes the following steps: S1: Obtain the lidar observation data of the UAV at the current moment, and calculate the proportion of unknown cue beams in multiple different directions based on the lidar observation data to obtain the unknown proportion in each direction; S2: Obtain the current motion command of the UAV, and calculate the effective motion components in the multiple different directions based on the current motion command; S3: Based on the unknown proportions in each direction and the effective motion components, construct and calculate the continuous frontal risk value at the current moment to quantify the potential risk of the UAV advancing toward the unknown area; S4: Perform time-recursive smoothing on the continuous frontier risk values ​​to obtain the memory risk value at the current moment; S5: Based on the comparison result between the memory risk value and at least one preset risk threshold, the discrete frontier risk level at the current moment is determined; S6: Based on the discrete frontier risk level, the dynamic penalty weight at the current moment is dynamically matched and obtained through the preset mapping relationship between risk level and penalty weight; S7: Based on the dynamic penalty weight and the memory risk value, construct an adaptive frontier risk penalty term, and use the adaptive frontier risk penalty term to comprehensively evaluate at least one candidate local navigation target, and select the local navigation target at the current time from the at least one candidate local navigation target; S8: Input the current state, which includes the continuous frontier risk value, the memory risk value, the discrete frontier risk level, the local navigation target, and the current environmental perception information and motion state information of the UAV, into the pre-trained reinforcement learning policy network to output the current action command for controlling the flight of the UAV.

2. The method for safe navigation in unknown environments based on dynamic weighting of frontier risks according to claim 1, characterized in that, In step S1, the proportion of unknown cue beams in multiple different directions is calculated based on the lidar observation data to obtain the unknown proportion in each direction, specifically including: The multiple laser beams in the lidar observation data are divided into forward, left, and right regions according to the body coordinate system. For each region, the number of laser beams with a measurement distance close to the maximum range of the lidar is counted, and the ratio of this number to the total number of laser beams in the region is taken as the proportion of unknown direction for the corresponding region.

3. The method for safe navigation in unknown environments based on dynamic weighting of frontier risks according to claim 1, characterized in that, In step S3, based on the unknown proportions in each direction and the effective motion components, the continuous frontier risk value at the current moment is constructed and calculated, specifically including: The weighted sum of the unknown proportions of the forward, left, and right directions and the effective motion components is used as the continuous frontier risk value.

4. The method for safe navigation in unknown environments based on dynamic weighted frontier risks according to claim 1, characterized in that, In step S4, the step of performing time-recursive smoothing on the continuous frontier risk values ​​to obtain the memory risk value at the current moment specifically includes: Obtain the memory risk value from the previous moment; Based on a preset risk decay coefficient, an exponentially weighted moving average is calculated on the memory risk value of the previous moment and the continuous frontier risk value of the current moment, and the result is used as the memory risk value of the current moment.

5. The method for safe navigation in unknown environments based on dynamic weighting of frontier risks according to claim 1, characterized in that, In step S5, the step of classifying the discrete frontier risk level at the current moment based on the comparison result between the memory risk value and at least one preset risk threshold specifically includes: When the memory risk value is less than or equal to the first preset threshold, it is determined to be a low risk level; When the memory risk value is greater than the first preset threshold and less than or equal to the second preset threshold, it is determined to be at a medium risk level; When the memory risk value is greater than the second preset threshold, it is determined to be a high-risk level.

6. The method for safe navigation in unknown environments based on dynamic weighting of frontier risks according to claim 1, characterized in that, In step S6, the step of dynamically matching the dynamic penalty weight at the current moment based on the discrete frontier risk level and through a preset mapping relationship between risk level and penalty weight specifically includes: A first penalty weight, a second penalty weight, and a third penalty weight are preset to correspond one-to-one with low-risk level, medium-risk level, and high-risk level, wherein the first penalty weight, the second penalty weight, and the third penalty weight increase sequentially; When the discrete frontier risk level is low, the first penalty weight is matched as the dynamic penalty weight; when the discrete frontier risk level is medium, the second penalty weight is matched as the dynamic penalty weight; when the discrete frontier risk level is high, the third penalty weight is matched as the dynamic penalty weight. In step S7, the adaptive frontier risk penalty term is obtained by calculating the product of the dynamic penalty weight and the memory risk value.

7. The method for safe navigation in unknown environments based on dynamic weighted frontier risks according to claim 1, characterized in that, In step S7, the step of comprehensively evaluating at least one candidate local navigation target using the adaptive frontier risk penalty term and selecting the local navigation target at the current moment from the at least one candidate local navigation target specifically includes: Obtain the basic benefit evaluation and unknown propulsion risk cost for each candidate target among at least one candidate local navigation target; For each candidate target, the comprehensive evaluation function value of the candidate target is obtained by subtracting the product of the adaptive frontier risk penalty term and the unknown propulsion risk cost from the basic benefit evaluation. The candidate target with the largest comprehensive evaluation function value is selected as the local navigation target at the current moment.

8. The method for safe navigation in unknown environments based on dynamic weighted frontier risks according to claim 1, characterized in that, In step S8, the input state of the pre-trained reinforcement learning policy network also includes the relative position or direction information of the local navigation target at the current moment; the current action command includes one or more of the UAV's desired speed, heading adjustment amount, or attitude correction amount.

9. The method for safe navigation in unknown environments based on dynamic weighted frontier risks according to claim 1, characterized in that, In step S8, the pre-trained reinforcement learning policy network adopts the proximal policy optimization algorithm and is trained using an immediate reward function; the immediate reward function includes a task advancement reward, a basic safety constraint reward, and an adaptive frontier risk penalty.

10. A safety navigation system for unknown environments based on dynamic weighted assessment of cutting-edge risks, characterized in that: include: The unknown proportion calculation module is used to acquire the lidar observation data of the UAV at the current moment, and calculate the proportion of unknown cue beams in multiple different directions based on the lidar observation data to obtain the unknown proportion in each direction; An effective motion component calculation module is used to obtain the current motion command of the UAV and calculate the effective motion components in multiple different directions based on the current motion command; The continuous front risk construction module is used to construct and calculate the continuous front risk value at the current moment based on the unknown proportions in each direction and the effective motion components, so as to quantify the potential risk of the UAV advancing toward the unknown area. The memory risk smoothing module is used to perform time-based smoothing on the continuous leading edge risk values ​​to obtain the memory risk value at the current moment. The risk level classification module is used to classify the discrete frontier risk level at the current moment based on the comparison result between the memory risk value and at least one preset risk threshold. The dynamic penalty weight matching module is used to dynamically match the dynamic penalty weight at the current moment based on the discrete frontier risk level and through a preset mapping relationship between risk level and penalty weight. The local navigation target selection module is used to construct an adaptive frontier risk penalty term based on the dynamic penalty weight and the memory risk value, and to use the adaptive frontier risk penalty term to comprehensively evaluate at least one candidate local navigation target, and select the local navigation target at the current time from the at least one candidate local navigation target; The action command output module is used to input the current state, which includes the continuous front risk value, the memory risk value, the discrete front risk level, the local navigation target, and the current environmental perception information and motion state information of the UAV, into a pre-trained reinforcement learning policy network, so as to output the current action command for controlling the flight of the UAV.

Images