Unmanned aerial vehicle image target navigation reinforcement learning safety control method, device and equipment

Abstract

The present application relates to the technical field of unmanned aerial vehicle vision navigation and intelligent control, and provides an unmanned aerial vehicle image target navigation reinforcement learning safety control method, device and equipment aiming at the problems of vision forgetting, exploration deadlock and insufficient safety guarantee in existing reinforcement learning navigation.The method comprises the following steps: adopting a performance-driven periodic level entropy self-adaptive adjustment mechanism, dynamically adjusting the entropy weight coefficient based on a comprehensive performance evaluation index to break through local optimization; adopting a smoothing truncation mechanism, constructing a proxy loss based on a smoothing truncation operator to maintain the consistency of the visual encoder features; constructing a kinematics safety constraint based on depth driving, deriving and dynamically activating a linear inequality; performing risk perception action filtering and conflict processing, solving the minimum invasive safety action by quadratic programming and responding to the solution failure.The present application realizes the cooperation of adaptive optimization and deterministic safety guarantee of the navigation strategy, and significantly improves the navigation efficiency and flight safety of the unmanned aerial vehicle in a mapless environment.

Description

Technical Field

[0001] This invention belongs to the field of UAV visual navigation and intelligent control technology, specifically relating to a UAV image target navigation reinforcement learning safety control method, device and equipment. Background Technology

[0002] Visual navigation (VN) technology is a core support for the autonomous control and perception system of UAVs. Its autonomous obstacle avoidance and path planning capabilities directly determine the safety and timeliness of UAVs in carrying out tasks such as material delivery and environmental surveying in complex and unknown environments. Among them, image-goal navigation (ImageNav) guides UAVs to find specific locations in extreme environments without maps or GPS signals by using target images. This endows UAVs with environmental adaptability and target search capabilities in key scenarios such as disaster relief and industrial inspection, and has become a research hotspot in the field of autonomous UAV navigation.

[0003] However, due to the inherent hard gradient truncation characteristic of existing Deep Reinforcement Learning (DRL) algorithms, existing image navigation methods are prone to gradient discontinuities during training, leading to catastrophic forgetting of visual features. Although strategies such as Proximal Policy Optimization (PPO) can maintain stability by limiting the update magnitude, their static entropy reward and penalty mechanisms struggle to provide flexible exploration boundaries when facing visually homogenized scenes or environmental noise, making UAVs highly susceptible to exploration deadlock. Regarding underlying obstacle avoidance safety, existing probabilistic neural correction-based methods face severe challenges: on the one hand, simplifying safety to soft reward constraints or probabilistic penalties makes it difficult to provide deterministic physical boundaries in complex geometric environments, resulting in significantly increased trajectory oscillations for UAVs in environments such as narrow passages; on the other hand, in unknown scenarios such as distributed external environments, obstacle avoidance models that highly rely on data distribution struggle to provide rigorous zero-collision guarantees, severely impacting the system survivability and mission execution quality of UAVs in actual deployments.

[0004] To alleviate this problem, researchers have proposed various improvement strategies. In terms of perceptual representation, some methods introduce memory enhancement mechanisms or topological graph models to enhance the robustness of visual routing by constructing semantic associations of the environment or global spatial mapping. These methods improve the model's utilization of historical observation information to some extent, but when dealing with completely unfamiliar open scenes, they are often limited by pre-defined topological structures or stringent category matching logic, lacking sufficient generalization ability. In recent years, with the development of multimodal pre-training technology, some research has attempted to project cue words and visual features into a unified semantic space, achieving target-oriented reasoning in zero-shot scenarios and improving search efficiency in open scenes. However, such end-to-end algorithms still fail to address the continuity of the underlying optimization objective; under high-dimensional visual flow input, the instability of gradient updates still leads to slow policy convergence. Regarding safety assurance, existing mainstream frameworks attempt to compress wide-field-of-view depth sensor data, transforming it into collision probabilities or heuristic safety penalties. These methods learn environmental geometric features through neural networks, aiming to endow UAVs with autonomous obstacle avoidance capabilities. However, this approach of abstracting high-dimensional geometric boundaries into low-dimensional probabilistic signals inherently weakens the deterministic physical boundaries provided by depth cameras. In actual flight, due to the perceptual bias of neural networks towards out-of-distribution scenes, this probabilistic correction mechanism is highly susceptible to inducing frequent course jitter and trajectory oscillations, failing to meet stringent flight safety requirements in confined spaces or environments with dense obstacles. Furthermore, while some sampling-based local path planning algorithms can alleviate obstacle avoidance pressure, a trade-off between computational overhead and navigation efficiency is difficult to achieve without a map. Therefore, how to construct a safety control mechanism that combines mathematical provability and physical determinism while ensuring the stability of high-dimensional visual feature extraction and efficient exploration capabilities remains a pressing challenge in the field of UAV autonomous navigation.

[0005] Therefore, in unknown and complex environments, how to achieve continuous and stable learning of visual features and balance the efficiency of active exploration with deterministic physical obstacle avoidance is crucial for improving the exploration efficiency and underlying flight safety of image target navigation systems in mapless and GPS-denied scenarios. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a reinforcement learning-based safety control method, apparatus, and device for UAV image target navigation. In UAV image target navigation scenarios, without relying on prior environmental maps or introducing additional supervisory signals, the method uses flight interaction performance and physical safety boundaries as joint constraints. It dynamically coordinates exploration and utilization through a performance-driven periodic entropy adaptive adjustment mechanism, eliminates gradient mutations in the visual encoder through a smoothing truncation strategy to maintain feature representation consistency, directly maps raw depth information to deterministic linear safety constraints through a depth-driven control obstacle function, and achieves minimally invasive coordination between navigation intent and safety constraints through risk-aware action filtering and conflict handling. This ensures synergistic optimization of entropy weight coefficients, visual feature representations, safety action corrections, and final navigation success rate and flight safety, while mitigating the probabilistic misjudgment risk of pure data-driven obstacle avoidance strategies in dense obstacle areas, thereby improving the autonomy and reliability of UAV image target navigation in map-less complex environments.

[0007] This invention provides a reinforcement learning-based safety control method for UAV image target navigation, comprising: Step S110: During the training phase, an improved proximal policy optimization algorithm is used to train the policy network of navigation reinforcement learning. When updating the policy, the improved proximal policy optimization algorithm uses a smooth adjustment mechanism with first-order derivative continuity at the truncation boundary to replace the hard truncation function, and combines an entropy weight coefficient based on multi-dimensional performance feedback adaptive adjustment to construct a surrogate loss function to maintain the consistency of the feature representation of the visual encoder. Step S120: Input the environmental perception information and the reference action output by the navigation reinforcement learning policy network into a pre-trained risk assessment network to predict the collision probability prediction value of performing the reference action. Step S130: Using the original depth image acquired by the airborne depth sensor, construct depth-driven kinematic safety constraints: construct discrete observation vectors by segmenting minimum depth sampling; derive linear safety constraint inequalities for the UAV control vector based on the control obstacle function theory; and dynamically activate the corresponding constraints according to the depth sampling values. Step S140: When the predicted collision probability exceeds a preset risk threshold, an optimal safe action is solved under the constraint set formed by the linear safety constraint inequalities, with the reference action as the optimization target, and the optimal safe action is output as the actual action to be executed; otherwise, the reference action is directly output as the actual action to be executed.

[0008] On the other hand, the present invention also provides a UAV image target navigation reinforcement learning safety control device, comprising: The first module is used to train the navigation reinforcement learning policy network using an improved proximal policy optimization algorithm during the training phase. The improved proximal policy optimization algorithm replaces the hard truncation function with a smooth adjustment mechanism that has the continuity of the first derivative at the truncation boundary during policy update. It combines an entropy weight coefficient based on multidimensional performance feedback to construct a surrogate loss function to maintain the consistency of the feature representation of the visual encoder. The second module is used to input environmental perception information and the reference action output by the policy network of the navigation reinforcement learning into a pre-trained risk assessment network to predict the collision probability prediction value of performing the reference action. The third module is used to construct depth-driven kinematic safety constraints using the raw depth images acquired by the airborne depth sensor: a discrete observation vector is constructed by segmenting the minimum depth sampling; a linear safety constraint inequality for the UAV control vector is derived based on the control obstacle function theory; and the corresponding constraints are dynamically activated based on the depth sampling values. The fourth module is used to solve for an optimal safe action under the constraint set formed by the linear safety constraint inequalities when the collision probability prediction value exceeds the preset risk threshold, with the reference action as the optimization target, and output the optimal safe action as the actual action; otherwise, the reference action is directly output as the actual action.

[0009] Compared with the prior art, the beneficial effects obtained by the present invention include: (1) Through the performance-driven periodic entropy adaptive adjustment mechanism, exploration noise is dynamically injected or attenuated according to the actual flight performance. It can actively break through the local optimum when the strategy stagnates and make stable use when the strategy converges, thus achieving a dynamic balance between exploration and utilization. This effectively improves the navigation pass rate in homogeneous scenarios and solves the exploration deadlock problem in the existing technology.

[0010] (2) To address the problem of catastrophic forgetting of visual features in existing technologies, a smooth truncation mechanism is used to eliminate gradient abrupt changes caused by hard truncation, enabling the visual encoder to receive continuous and stable gradient signals during policy updates, protecting the learned obstacle semantics and target feature representations, and enhancing the generalization ability and convergence stability of the policy in cross-domain environments.

[0011] (3) By constructing kinematic safety constraints driven by depth, the original depth information is directly transformed into linear hard constraints based on the control obstacle function, which mathematically ensures that the flight trajectory does not intrude into the preset safety area and provides a deterministic geometric safety barrier.

[0012] (4) By filtering and handling risk perception actions, the minimum intrusive correction actions are solved by quadratic programming, the navigation intention of high-level intelligent decision-making is preserved to the greatest extent, and the retreat and backtracking strategy is executed when constraints are not feasible, so as to ensure flight safety in complex topology environments and solve the conflict coordination between navigation intention and safety constraints.

[0013] This invention achieves organic synergy between adaptive optimization of navigation strategy and deterministic safety assurance through a perception and decision framework that decouples high and low layers, significantly improving the navigation success rate and robustness of virtual-to-real migration of UAVs in complex environments without maps. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating the steps of a UAV image target navigation reinforcement learning safety control method in one embodiment of the present invention; Figure 2 This is a schematic diagram of the performance-driven entropy adaptive adjustment algorithm in one embodiment of the present invention; Figure 3 This is a schematic diagram of the calculation process of the smooth truncation mechanism in one embodiment of the present invention; Figure 4 This is a schematic diagram of a CBF constraint model based on depth information in one embodiment of the present invention, wherein, Figure 4 (a) is a schematic diagram of the normalization results of depth image data. Figure 4 (b) is a schematic diagram of the minimum depth sampling in the sampling column of the depth image. Figure 4 (c) is a polar view of the obstacle distribution centered on the drone; Figure 5 This is a schematic diagram of the action filtering and conflict handling algorithm in one embodiment of the present invention; Figure 6 This is a complete flowchart of a UAV image target navigation reinforcement learning safety control method according to one embodiment of the present invention; Figure 7 This is a schematic diagram of the three-dimensional flight trajectory of the UAV in the experiment of this invention, colored according to the magnitude of its instantaneous velocity; Figure 8 This is a schematic diagram of the three-dimensional flight trajectory of the UAV in the experiment of this invention as a two-dimensional top-view projection on the XY plane. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] In one embodiment, such as Figure 1 As shown, the present invention provides a reinforcement learning-based safety control method for UAV image target navigation, comprising: Step S110: During the training phase, an improved proximal policy optimization algorithm is used to train the policy network of navigation reinforcement learning. When updating the policy, the improved proximal policy optimization algorithm uses a smooth adjustment mechanism with first-order derivative continuity at the truncation boundary to replace the hard truncation function, and combines an entropy weight coefficient based on multi-dimensional performance feedback adaptive adjustment to construct a surrogate loss function to maintain the consistency of the feature representation of the visual encoder. Step S120: Input the environmental perception information and the reference action output by the navigation reinforcement learning policy network into a pre-trained risk assessment network to predict the collision probability prediction value of performing the reference action. Step S130: Using the original depth image acquired by the airborne depth sensor, construct depth-driven kinematic safety constraints: construct discrete observation vectors by segmenting minimum depth sampling; derive linear safety constraint inequalities for the UAV control vector based on the control obstacle function theory; and dynamically activate the corresponding constraints according to the depth sampling values. Step S140: When the predicted collision probability exceeds a preset risk threshold, an optimal safe action is solved under the constraint set formed by the linear safety constraint inequalities, with the reference action as the optimization target, and the optimal safe action is output as the actual action to be executed; otherwise, the reference action is directly output as the actual action to be executed.

[0018] Specifically, in step S110: performance-driven periodic entropy adaptive adjustment.

[0019] Existing UAV image target navigation methods based on Proximal Policy Optimization (PPO) often face severe oscillations in surrogate loss when processing high-dimensional visual input. Traditional fixed entropy weighting mechanisms cannot balance the contradiction between escaping spatial deadlock and maintaining smooth flight, easily causing UAVs to get stuck in homogeneous scenarios such as dead ends with highly repetitive visual textures.

[0020] To address this issue, this invention designs a performance-driven periodic entropy adaptive adjustment mechanism. It employs a comprehensive performance index reflecting the actual interaction between the UAV and its environment to dynamically guide the exploration strategy. Specifically, it constructs a surrogate loss function by designing entropy weight coefficients based on multi-dimensional performance feedback adaptive adjustment to maintain the consistency of the visual encoder's feature representations and to explore and utilize the dynamic balancing strategy. The performance-driven entropy adaptive adjustment algorithm flow is as follows: Figure 2 As shown. The specific implementation steps are as follows: 1) Calculation of comprehensive performance evaluation indicators At the end of each reinforcement learning training epoch, the system first collects the data for the current reinforcement learning training epoch. The flight trajectory interaction data of the UAV within a certain period is collected and stored in a trajectory buffer. Subsequently, data is read from the trajectory buffer to calculate three dimensions of environmental interaction evaluation indicators: the average reward per period, representing the target-oriented capability. Step reward variance characterizing flight instability and motion deviations that characterize trajectory smoothness .

[0021] In one embodiment, the motion deviation Take the L2 norm deviation of adjacent actions.

[0022] To filter out the instantaneous numerical instability caused by high-dimensional visual input, this invention introduces an exponential moving average (EMA) algorithm to maintain a smooth baseline value for each environmental interaction evaluation index. For any environmental interaction evaluation index... Its EMA update logic is shown in formula (1): (1) in, Indicates the first Environmental interaction evaluation indicators for each cycle The smoothing baseline value, The set smoothing factor, For the first The instantaneous observation values ​​of the periodic evaluation index are used. Based on the above smoothed benchmark values, in order to strictly punish irregular and violent movements while encouraging high-reward behavior, this invention regards the normalized average reward as a positive gain, variance as a negative indicator, and action deviation as a penalty term. Thus, a comprehensive performance evaluation index strictly constrained by physical feasibility is constructed. Its calculation is shown in formula (2): (2) in, Indicates the first The overall performance score for each cycle; , and These represent the average reward, step reward variance, and motion trajectory smoothness for the current period, respectively. , and These are the corresponding smoothing baseline values; , , The preset weighting coefficients are (in this invention, they are set to 0.6, 0.3, and 0.1, respectively).

[0023] In one embodiment, the smoothing factor is set to 0.99.

[0024] In one embodiment, the preset weighting coefficients are respectively taken as , , .

[0025] 2) Dynamic adjustment of entropy weights based on performance changes: After obtaining the comprehensive performance evaluation index, the system calculates the change in the comprehensive performance evaluation index between the current period and the previous period. The system adaptively and dynamically adjusts the next cycle (the 1st cycle) based on the positive or negative attribute of this change. Entropy weight coefficient (per cycle) This is to achieve a seamless switching between "radical breakthrough" and "stable utilization". The specific adjustment judgment conditions and execution logic are shown in formula (3): (3) in, Defined as amplification factor, Defined as attenuation factor, For the first The entropy weight coefficients for each period. Simultaneously, the system sets the physical cutoff boundary for the entropy weights. To prevent numerical overflow or excessive collapse, The upper limit of entropy weight. This is the lower bound of the entropy weight. Specifically: if This means the system detects a continuous decline or stagnation in navigation performance, indicating that the drone is highly likely to be stuck in a space exploration deadlock. At this point, the system triggers the amplification factor. By abruptly increasing the entropy weights, strong exploration noise is injected into the action space, forcing the UAV to perform aggressive maneuvers to escape the current local optimum; conversely, if This indicates that the drone has found an effective obstacle avoidance strategy and its path toward the target is gradually stabilizing, at which point the system triggers the decay factor. By smoothly reducing entropy weights, the network converges and prioritizes the use of the current optimal flight trajectory, thus avoiding excessive and disordered exploration from interfering with lower-level flight stability. This mechanism dynamically injects or attenuates exploration noise based on actual flight performance during ascents and descents. It proactively breaks through local optima when the strategy stagnates and stably utilizes the current trajectory when the strategy converges, achieving a dynamic balance between exploration and utilization. This effectively improves navigation throughput in homogeneous scenarios and solves the exploration deadlock problem.

[0026] In one embodiment, the amplification factor is set to .

[0027] In one embodiment, the attenuation factor is set to .

[0028] In one embodiment, the upper bound and lower bound of the entropy weight are respectively set as follows: , .

[0029] Next, in step S120: feature consistency maintenance based on smooth truncation. In ImageNav tasks, the environmental features extracted by the onboard visual encoder are crucial for the UAV's semantic understanding and obstacle avoidance decisions. However, conventional PPO algorithms employ a hard clipping mechanism during policy updates, which causes the probability ratio to remain within the truncation boundary (i.e., ... The first derivative at point () is discontinuous. This discontinuous gradient, when backpropagated to the high-dimensional visual encoder (using ResNet-9 in this invention), easily disrupts the learned spatial feature representation, leading to a "catastrophic forgetting" phenomenon. To maintain the consistency of visual features and ensure the stability of policy updates, this invention designs a method with... The continuous smooth clipping update mechanism, the computational process of the smooth clipping mechanism is as follows: Figure 3 As shown. The specific process is as follows: 1) Smooth Truncation Operator The segmented construction is as follows: First, based on the policy parameter update, the probability ratio between the current policy and the old policy is defined as... To establish a smooth gradient buffer region at the truncation boundary, this invention constructs a smoothing truncation operator using a piecewise polynomial form. By introducing a quadratic smoothing term at the hard truncation boundary, abrupt gradient changes are eliminated. For a preset smoothing truncation threshold... The mathematical expression for the smooth truncation operator is shown in formula (4): (4) in, Indicates the cutoff boundary (of the right buffer segment). The upper bound of the constructed quadratic polynomial buffer function, Indicates the cutoff boundary (of the left buffer segment). The lower bound quadratic polynomial buffer function constructed at that point. The preset smooth cutoff threshold; It is the probability ratio between the current strategy and the old strategy. These are the current policy network parameters (to be optimized). It is the conditional probability of the current policy. It is the conditional probability of the old strategy. These are the old strategy network parameters, usually obtained before the start of this optimization round. The copied snapshot remains unchanged in this round of updates; Indicates a time step; The state is represented by a vector, describing the state at time step. The state that the agent observes from the environment; Indicates an action, Indicates at time step The agent is in the state according to the old policy. The actions actually performed under the given conditions. In the intermediate linear segment. Define linear form Upper / lower bound quadratic polynomial buffer functions are constructed at different truncation boundaries to match the first derivative at the boundary points to ensure gradient continuity.

[0030] In one embodiment, the preset smoothing cutoff threshold is set to .

[0031] 2) Determination of the derivative continuity condition: To ensure the continuity of the gradient flow during gradient backpropagation, the above-mentioned quadratic polynomial buffer term... and The zero-order value at the boundary must be consistent with the first-order derivative, which means that the boundary derivatives have a continuous and smooth transition.

[0032] The upper bound quadratic polynomial buffer function The definition satisfies: , ; The lower bound quadratic polynomial buffer function The definition satisfies: , .

[0033] Boundary points For example, buffer function The function value at that point must be equal to... Furthermore, the slope at this point is strictly equal to 1. By calculating the matching coefficient, this smoothing operator enables the gradient to decay gradually when the probability ratio exceeds the cutoff range, rather than instantaneously zeroing out. This mechanism provides a smooth parameter update buffer for the visual encoder, effectively mitigating the large oscillations in visual feature layer parameters caused by drastic changes in action space probabilities, thereby ensuring the long-term consistency of feature representation.

[0034] 3) Integration of the smoothed surrogate loss function: combining the entropy weight coefficients dynamically obtained in step S110 This invention further constructs the final feature consistency maintenance proxy loss function. This is used to guide the parameter optimization of the visual navigation model. The definition of this proxy loss function is shown in formula (5): (5) in, This represents the total loss function to be optimized; This represents the empirical expectation of a time step; This represents an estimate of the advantage function, an intermediate value output by the value network (Critic); The entropy represents the distribution of the policy network output. This represents the conditional probability of the current policy. Indicates at time step Observational data on the state.

[0035] When performing gradient descent updates, in equation (5) The operator ensures that the first-order gradient of the objective function with respect to the policy network parameters is continuous across the entire domain. This allows the visual encoder to receive stable and continuous semantic feedback signals even when the UAV performs large maneuvers such as obstacle avoidance (i.e., when the probability ratio fluctuates significantly). This mechanism effectively mitigates catastrophic forgetting, solving the problem of catastrophic forgetting in reinforcement learning for visual tasks from the algorithm's underlying layer. This enables the UAV to maintain its ability to recognize obstacle features and target semantics over a long period, significantly enhancing the generalization performance and convergence quality of the policy in cross-domain environments and complex physical scenarios. Therefore, the smooth truncation mechanism eliminates the gradient abrupt changes caused by hard truncation, allowing the visual encoder to receive continuous and stable gradient signals during policy updates, protecting the learned obstacle semantics and target feature representations, and enhancing the policy's generalization ability and convergence stability in cross-domain environments.

[0036] Further, in step S130, depth-driven kinematic safety constraints are constructed.

[0037] To overcome the probabilistic limitations of data-driven neural networks in ensuring safety, this invention utilizes real-time acquired depth image data, combined with Control Barrier Function (CBF) theory, to construct a deterministic geometric safety barrier for the UAV. This step aims to directly transform high-dimensional visual perception into underlying kinematic hard constraints, thereby mathematically guaranteeing the safety of the flight trajectory. The specific implementation process is as follows: 1) Perform local barrier function modeling based on depth sampling: acquire raw depth images within the field of view (FoV) range in real time using an airborne depth sensor. To reduce computational complexity while preserving the integrity of obstacle geometry, this embodiment performs column-based minimum depth sampling on the original depth image. First, the original depth image is divided horizontally into... There are 1 sampling column, each covering a certain angle range, where the 1st sampling column will be... The sampling rays in the sampling column (referred to as the first) The angle of the sampling ray (the angle of deviation of the sampling ray relative to the UAV's heading) is used as the sampling ray angle. For each sampling column, the minimum depth value among all depth pixels within it is taken as the depth sampling value of that sampling column. This ultimately forms the discrete observation vector in polar coordinates. Each sampling point corresponds to a specific sampling ray angle. Let the physical safety margin be... Regarding the first Sampling rays, constructing local barrier functions As shown in formula (6): (6) Among them, the local barrier function Scalar values ​​used for making safety determinations; This is a preset physical safety margin (or safety threshold).

[0038] In one embodiment, Set to 0.30m.

[0039] According to the forward invariance (CBF) theory, to ensure that the drone never enters the collision zone centered on the obstacle, the "forward invariance" of the safety set must be ensured, which requires the control input to... Satisfying Lie derivative hard constraints , used to control the local barrier function; where , This is a preset safety adjustment coefficient used to adjust the stringency of the safety boundary.

[0040] 2) Kinematic Correlation and Linear Constraint Derivation: The planar motion of the UAV is modeled as a continuous nonlinear affine control system. Based on the UAV's planar kinematic model and the described relative kinematic relationship between the UAV and the obstacle, the derivative of the local barrier function with respect to time is established (due to...). It is a constant quantity, and is also equal to the depth measurement value. rate of change over time ) and the linear velocity of the drone and angular velocity They exhibit a strong coupling relationship, and their derivative expression is shown in formula (7): (7) in, , They represent respectively to , Find the first derivative.

[0041] Substituting formula (7) into the Lie derivative hard constraint in the CBF safety criterion, and rearranging the terms, we can derive the information about the control variables (vectors). The linear safety constraint inequality is shown in equation (8): (8) in, Let be the control variable (vector) to be determined. This is a preset safety adjustment coefficient used to adjust the deceleration response sensitivity of the drone when it approaches an obstacle; This indicates the transpose operation. This inequality defines the feasible domain of actions that the drone is allowed to perform to avoid collisions under the current perception environment.

[0042] 3) Dynamically Activating Corresponding Constraints Based on Depth Sample Values: Considering the redundant computational overhead of calculating constraints for all sample points in an open space, this invention designs a dynamic activation mechanism based on activation thresholds. The system monitors the depth sample values ​​of each sample column in real time. Set activation threshold ,in, A positive preset offset; only when Less than the activation threshold When the sampling ray is sampled, the corresponding linear safety constraint inequality is incorporated into the linear safety constraint set of the subsequent action filtering module (quadratic programming solver).

[0043] This mechanism ensures that the drone can maintain its high-altitude intelligent exploration characteristics when it is far from obstacles, while it can quickly switch to a strict physical obstacle avoidance mode when it approaches a risky area, effectively balancing navigation efficiency and safety. Figure 4 For a CBF constraint model based on depth information, where, Figure 4 (a) is a schematic diagram of the normalization result of depth image data. The horizontal axis is the image width and the vertical axis is the image height. The unit is pixels. Figure 4 (b) is a schematic diagram of the minimum depth sampling in the sampling column of the depth image. The horizontal axis is the sampling column index (representing the column index / ray number of different angle ranges), and the vertical axis is the normalized depth value. Figure 4 (c) is a polar view of the obstacle distribution centered on the UAV. This is a polar coordinate representation of the environment surrounding the UAV, with the center of the circle representing the current position of the UAV, the polar radius direction representing the ray direction from the UAV to the obstacle, and the polar radius length representing the distance to the obstacle in that direction, in meters. Thus, the original depth information is directly transformed into linear hard constraints based on the obstacle control function, mathematically ensuring that the flight trajectory does not intrude into the preset safety area, providing a deterministic geometric safety barrier.

[0044] In one embodiment, the positive preset offset is taken as .

[0045] Finally, in step S140, risk-aware action filtering and conflict handling are performed. This aims to preserve the intelligent navigation intent of higher-level strategies to the greatest extent possible while ensuring physical safety through a risk-driven triggering mechanism. When environmental complexity exceeds the processing limit of the local planner, the system ensures the survivability of the UAV through a conflict handling mechanism. The specific implementation process is as follows: 1) A risk assessment and triggering mechanism based on deep features is used to predict the collision probability of the reference action output by the high-level navigation model through a risk assessment network.

[0046] Before executing the action, the system first uses a risk assessment network to evaluate the reference actions output by the high-level navigation model. Real-time security evaluation is performed. This step utilizes Convolutional Neural Networks (CNNs) to extract depth images. Potential spatial features ; potential spatial features Reference motion output by the high-level navigation model After being stitched together, the data is input into a multi-layer perceptron (MLP) to predict the current collision probability, thus obtaining the predicted collision probability value. The risk assessment follows the triggering logic shown in formula (9): (9) in, Indicates at time step The actual actions executed by the drone, i.e., the safety actions; This indicates a safe action corrected by the quadratic programming solver; A preset risk threshold is set; when the predicted collision probability is lower than the threshold, the drone directly executes high-level reference actions to maintain navigation efficiency; once the risk exceeds the threshold, a deterministic safety filter is activated to correct the actions.

[0047] In one embodiment, the preset risk threshold .

[0048] 2) Minimal intrusive action correction based on quadratic programming (QP): When the predicted collision probability exceeds the preset risk threshold, the reference action is input into the quadratic programming solver, and the safe action is solved and output under the linear safety constraints.

[0049] Once the safety filter is triggered, the system models the action correction process as a constrained quadratic programming (QP) problem. The core of this problem is to find a reference action that satisfies the linear safety constraints derived in step S130. The safest action with the smallest Euclidean distance The mathematical model of the quadratic programming solver is described in the form shown in equation (10): (10) st , ; in, The objective function embodies the principle of minimal intrusion into the intent of higher-level intelligence. This represents the set of linear security constraints generated from all activated sampling points in step S130. and Let represent the coefficient matrix and constant vector formed by the linear security constraint inequalities, respectively; and Physical limits are imposed on the drone actuators; control obstacle function constraints include: , .

[0050] In one embodiment, the linear velocity is limited to the range of [0, 3.0] m / s, and the angular velocity is limited to the range of [-15.0, 15.0] ° / s.

[0051] 3) When the quadratic programming solver fails to solve the problem, the rollback and recovery mechanism in the conflict scenario is executed to guide the UAV to avoid obstacles in an emergency.

[0052] In extremely narrow or dense topological environments, multiple CBF constraints may cause the feasible region of the QP problem to shrink to an empty set (i.e., QP has no solution). To address such conflict scenarios, this embodiment designs a conflict handling scheme that couples forced fallback with reversal. When QP solution failure (Infeasible) is detected, the system immediately interrupts optimization and switches to fallback mode, executing the fallback obstacle avoidance command as shown in formula (11): (11) in, The preset forced reverse speed, This is the emergency turn rate. The system simultaneously assesses the furthest detection range of rays on both sides of the airborne field of view (FoV) and guides the UAV to rotate to the relatively open side for emergency obstacle avoidance. This deterministic remediation strategy effectively solves the navigation trap problem caused by limitations in local perception, ensuring the system's survivability and robustness under extreme conditions. Figure 5 This is a flowchart of the action filtering and conflict resolution algorithm. This mechanism uses quadratic programming to solve for the least intrusive corrective actions, preserving the navigation intent of higher-level intelligent decisions to the greatest extent possible. When constraints are infeasible, a backtracking strategy is executed to ensure flight safety in complex topology environments, resolving the conflict and coordination between navigation intent and safety constraints.

[0053] In one embodiment, the preset forced backward speed is taken as... The emergency turning angular velocity is dynamically determined based on the openness on both sides of the field of view. .

[0054] In summary, this invention addresses the three major shortcomings of existing UAV image target navigation systems based on deep reinforcement learning: catastrophic forgetting of visual features, exploration deadlock caused by static entropy mechanism, and lack of deterministic safety guarantees in pure data-driven obstacle avoidance. It proposes a complete solution with deep coupling of training and execution. Figure 6 A complete flowchart of the UAV image target navigation reinforcement learning safety control method provided by this invention is presented. The complete working framework includes four parts: input module, navigation model, obstacle avoidance model, and deployment module. (1) In the input module, the target image is the visual target to be reached in the navigation task, the current observation (image) is the color image acquired in real time by the UAV's onboard RGB-D camera, and the depth observation refers to the original depth image. This is used for subsequent depth sampling and control barrier function (CBF) constraint construction.

[0055] (2) In the navigation model (reinforcement learning navigation): ResNet is used to extract features from the target image and the current observation respectively to obtain the target image features. Compared with current observation features Then, the two are merged by concatenation or element-by-element addition. The fused features are received using a gated recurrent unit (GRU). The temporal dependencies are modeled, and the hidden states are output for use by the policy network (Actor) and value network (Critic). Other temporal models such as LSTM and Transformer can also be used instead. The feature fusion and GRU-based temporal feature extraction described above are merely exemplary implementations to demonstrate the complete working process and are not essential technical processes required for the invention's protection scheme.

[0056] The core content protected by this invention is: based on the state representation output by a temporal feature extraction network (not limited to GRU, but can use LSTM, Transformer, or directly use the features of the current frame), an action probability distribution is generated, and a reference action is sampled. ,in Linear velocity, Angular velocity; output state value, used to calculate an estimate of the dominance function. Perform smooth truncation and define the probability ratio. By using the smooth truncation operator It replaces traditional hard truncation to ensure gradient continuity; it performs adaptive entropy adjustment by introducing performance-driven adaptive entropy weight coefficients. Dynamic equilibrium exploration and utilization; design of a proxy loss function for feature consistency maintenance. The truncated policy loss, value function error, and entropy regularization term are combined and used to update the network parameters.

[0057] (3) In the obstacle avoidance model: the original depth observation Perform minimum depth sampling by column to obtain discrete depth vectors; use a convolutional neural network (CNN) to extract the latent spatial features of the depth sampling. Using a multilayer perceptron (MLP), the CNN output is compared with the reference action. After splicing, it is mapped to the predicted collision probability value. Linear safety constraints for the control barrier function (CBF) are constructed based on depth sampling values; a quadratic programming solver is used with reference actions. To optimize the objective, the minimum intrusive safety action is solved under the CBF linear safety constraint and actuator physical limit. If the solution fails, a backtracking obstacle avoidance command will be executed.

[0058] (4) In the deployment module: when performing physical deployment, security actions will be implemented. The data is sent to the flight controller to drive the real drone in flight. During simulation deployment, safety actions are simultaneously sent to the simulated agent for training and verification. TensorBoard is used as an example of a visualization monitoring tool for the training process to complete training monitoring (such as recording losses and rewards). The monitoring tool TensorBoard is only used as an illustrative example and does not constitute a limitation on the training method of this invention.

[0059] Unlike existing technologies that employ hard gradient truncation, heuristic safety penalties, or independently operating safety filters, the core innovation of this invention lies in constructing a synergistic closed-loop system. At the high-level navigation layer, a performance-driven periodic entropy adaptive mechanism and a precise smooth truncation strategy work together. The former explores and utilizes dynamic balance based on multi-dimensional performance feedback, while the latter starts from the underlying optimization function to ensure the feature stability of the visual encoder during policy updates. Together, they significantly improve the training efficiency, convergence quality, and transfer capability from simulation to reality. At the low-level control layer, a kinematic control barrier function (CBF) based on raw depth information is constructed, directly mapping high-dimensional depth sampling to deterministic linear safety constraints. An innovative risk assessment network jointly sensing the environment and actions is introduced, triggering minimally invasive quadratic planning action corrections on demand. This ensures mathematically provable safety while maximizing the preservation of navigation intelligence in the high-level policy. Ultimately, through the organic integration of the above technical solutions, this invention effectively solves the problems of low navigation success rate, poor policy generalization ability, and insufficient safety in complex and unknown environments, achieving a significant improvement in navigation success rate and robustness of virtual-to-real transfer.

[0060] Furthermore, to verify the effectiveness of the method framework provided by this invention and its ability to transfer from simulation to real-world environments, this invention deploys the navigation strategy trained only in a simulation environment onto a physical quadcopter UAV platform to perform image target navigation tasks in an unstructured indoor environment of approximately 5.0 × 3.0 m. The UAV is equipped with an Intel RealSense D435i depth camera with a horizontal field of view of 87°. The acquired RGB-D images are downsampled to 128 × 128 and then transmitted to a ground station equipped with an NVIDIA GeForce RTX 4070. Unlike a fully airborne deployment, the ground station undertakes computationally intensive tasks such as policy network, risk assessment network, and safety action solution, and transmits speed commands back to the UAV at a closed-loop frequency of 10 Hz. The UAV is mainly responsible for image acquisition, data transmission, and flight execution; the entire system requires no external positioning or map-based planning assistance.

[0061] During the experiment, the high-level navigation strategy network outputs a reference action in real time based on visual input; simultaneously, the risk assessment network predicts the collision probability of executing the reference action. When the predicted collision probability does not exceed a preset risk threshold, the UAV directly executes the reference action; when the collision probability exceeds the risk threshold, the control obstacle function safety filter is triggered, using the reference action as the optimization target, and solving for a safe action under the linear kinematic safety constraints constructed in real time from the depth image, and outputting the safe action as the actual action to be executed.

[0062] like Figure 7 and Figure 8 As shown, Figure 7 A 3D flight trajectory diagram of a drone, colored according to its instantaneous velocity. Figure 8 This is a two-dimensional top-down projection of the three-dimensional flight trajectory onto the XY plane. Experimental results show that the actual total flight path length of the UAV from the starting point to the target point is approximately 4.8 m. During flight, the UAV can autonomously adjust its speed according to the distance to obstacles, decelerating to below 0.10 m / s when approaching foam pillar obstacles and accelerating to approximately 0.30 m / s when the path ahead is clear. This adaptive speed adjustment behavior is jointly achieved by the high-level navigation strategy and the low-level control obstacle function safety filter based on risk assessment results and condition-triggered correction, without the need for preset speed planning. The two-dimensional projection results show that the UAV produces a maximum lateral offset of approximately 0.7 m relative to the straight path, indicating that the strategy can actively generate continuous obstacle avoidance maneuvers instead of using a conservative stop-and-turn approach. The altitude change is controlled within ±0.3 m throughout the flight, and the minimum distance to obstacles during flight is 0.32 m, slightly higher than the set safety margin of 0.30 m, indicating that under real sensor noise conditions, the control obstacle function safety constraint can effectively maintain the safety envelope. No collisions occurred during the entire flight, indicating that the risk assessment network can effectively trigger the safety filter to intervene when needed.

[0063] In terms of real-time performance, the UAV-ground station closed-loop perception-control process, which includes image acquisition, downsampling and transmission, forward inference of policy network and risk assessment network, risk judgment, safety action solution and speed command feedback, has an end-to-end average latency of 48 ms, which meets the real-time requirement of 100 ms control cycle, thereby ensuring the stable closed-loop operation of the system in a real indoor environment.

[0064] In one embodiment, the present invention also provides a UAV image target navigation reinforcement learning safety control device, comprising: The first module is used to train the navigation reinforcement learning policy network using an improved proximal policy optimization algorithm during the training phase. The improved proximal policy optimization algorithm replaces the hard truncation function with a smooth adjustment mechanism that has the continuity of the first derivative at the truncation boundary during policy update. It combines an entropy weight coefficient based on multidimensional performance feedback to construct a surrogate loss function to maintain the consistency of the feature representation of the visual encoder. The second module is used to input environmental perception information and the reference action output by the policy network of the navigation reinforcement learning into a pre-trained risk assessment network to predict the collision probability prediction value of performing the reference action. The third module is used to construct depth-driven kinematic safety constraints using the raw depth images acquired by the airborne depth sensor: a discrete observation vector is constructed by segmenting the minimum depth sampling; a linear safety constraint inequality for the UAV control vector is derived based on the control obstacle function theory; and the corresponding constraints are dynamically activated based on the depth sampling values. The fourth module is used to solve for an optimal safe action under the constraint set formed by the linear safety constraint inequalities when the collision probability prediction value exceeds the preset risk threshold, with the reference action as the optimization target, and output the optimal safe action as the actual action; otherwise, the reference action is directly output as the actual action.

[0065] On the other hand, in one embodiment, the present invention provides a computer device including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the UAV image target navigation reinforcement learning safety control method provided in any of the above embodiments. The computer device may be a server. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device stores sample data. The network interface of the computer device is used for communication with external terminals via a network connection.

[0066] On the other hand, in one embodiment of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the UAV image target navigation reinforcement learning safety control method provided in any of the above embodiments.

[0067] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0068] Matters not covered in this invention are common knowledge.

[0069] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0070] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

[0071] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A reinforcement learning-based safety control method for image target navigation in unmanned aerial vehicles (UAVs), characterized in that, include: Step S110: During the training phase, the improved proximal policy optimization algorithm is used to train the policy network for navigation reinforcement learning. The improved near-end policy optimization algorithm replaces the hard truncation function with a smooth adjustment mechanism that has the continuity of the first derivative at the truncation boundary during policy update. It combines an entropy weight coefficient based on multidimensional performance feedback to construct a surrogate loss function to maintain the consistency of the feature representation of the visual encoder. Step S120: Input the environmental perception information and the reference action output by the navigation reinforcement learning policy network into a pre-trained risk assessment network to predict the collision probability prediction value of performing the reference action. Step S130: Using the original depth image acquired by the airborne depth sensor, construct a depth-driven kinematic safety constraint: construct a discrete observation vector by sampling the minimum depth in columns; Based on the control obstacle function theory, a linear safety constraint inequality for the UAV control vector is derived; Dynamically activate corresponding constraints based on depth sample values; specifically including: The raw depth image within the field of view of the airborne depth sensor is divided into multiple sampling columns along the horizontal direction, and the minimum depth value within each sampling column is taken as the corresponding sampling ray angle. depth sample value This constitutes the discrete observation vector in polar coordinates. ; Define local barrier function ,in, This is a preset physical safety margin; Based on the planar kinematic model of the UAV, the relationship between the derivative of the local barrier function with respect to time and the linear and angular velocities of the UAV is established: ； in, , They represent respectively to , Find the first derivative. and These represent the linear velocity and angular velocity of the drone, respectively. Substituting the above relationship into the Lie derivative hard constraint of the control local barrier function, we can... To obtain information about the control variables Linear security constraint inequalities: ； in, The preset safety adjustment coefficient, Indicates the transpose operation; Set activation threshold ,in, A positive preset offset; When the sampling ray angle Corresponding depth sampling value Less than When the sampling ray is sampled, the corresponding linear safety constraint inequality is included in the linear safety constraint set of the subsequent quadratic programming solver. Step S140: When the predicted collision probability exceeds a preset risk threshold, an optimal safe action is solved under the constraint set formed by the linear safety constraint inequalities, with the reference action as the optimization target, and the optimal safe action is output as the actual action to be executed; otherwise, the reference action is directly output as the actual action to be executed.

2. The UAV image target navigation reinforcement learning safety control method according to claim 1, characterized in that, In step S110, the process of obtaining the entropy weight coefficient based on multidimensional performance feedback adaptive adjustment includes: Collect flight trajectory interaction data within the current reinforcement learning training cycle and calculate three dimensions of environmental interaction evaluation metrics: cycle-average reward, step reward variance, and action bias, denoted as follows: , as well as ; The exponential moving average algorithm is used to maintain the smoothed baseline values ​​of the periodic average reward, step reward variance, and action deviation, respectively. Construct the comprehensive performance evaluation index according to the following formula: ； in, Indicates the first Comprehensive performance evaluation indicators for each cycle, , and These represent the smoothed baseline values ​​for the periodic average reward, the variance of the step reward, and the motion bias, respectively. , , Preset weighting coefficients; If the current number is The change in the comprehensive performance evaluation index between the current period and the previous period ,Will As the entropy weighting coefficient for the next cycle, where, As the amplification factor, , The upper limit of entropy weight. For the first Entropy weighting coefficients for each cycle; like ,Will As the entropy weighting coefficient for the next cycle, where, As the attenuation factor, , This is the lower bound of the entropy weight.

3. The UAV image target navigation reinforcement learning safety control method according to claim 1, characterized in that, In step S110, the smooth adjustment mechanism with first-order derivative continuity at the truncation boundary is implemented by constructing a smooth truncation operator in the form of a piecewise polynomial: ； in, To smooth the truncation operator, Indicates at the truncation boundary The upper bound of the constructed quadratic polynomial buffer function, Indicates at the truncation boundary The lower bound quadratic polynomial buffer function constructed at that point. The preset smooth cutoff threshold; It is the probability ratio between the current strategy and the old strategy. These are the current policy network parameters. It is the conditional probability of the current policy. It is the conditional probability of the old strategy. These are the old strategy network parameters. Indicates state, Indicates an action, It is a time step; The upper bound quadratic polynomial buffer function satisfy: ， ； The lower bound quadratic polynomial buffer function satisfy: ， 。 4. The UAV image target navigation reinforcement learning safety control method according to claim 3, characterized in that, In step S110, the proxy loss function is given by the following formula: ； in, This represents the expected value of the time step. This represents the estimated value of the advantage function. For the first Entropy weighting coefficients for each period, The entropy of the policy network output distribution is given. Indicates at time step Observational data.

5. The UAV image target navigation reinforcement learning safety control method according to claim 1, characterized in that, In step S120, the predicted collision probability value for performing the reference action includes: Extracting latent spatial features from raw depth images using convolutional neural networks ; latent spatial features Reference motion output by the high-level navigation model After stitching, the data is input into a multilayer perceptron, which outputs a predicted collision probability value. ; The triggering logic for risk assessment using the aforementioned risk assessment network is as follows: ； in, Indicates at time step The actual action commands executed by the drone This indicates a safe action corrected by the quadratic programming solver; The preset risk threshold; When the collision probability prediction value Greater than the preset risk threshold When the action is corrected, the quadratic programming solver is activated and the UAV executes the safety action corrected by the quadratic programming solver; otherwise, the UAV directly executes the reference action.

6. The UAV image target navigation reinforcement learning safety control method according to claim 5, characterized in that, In step S140, solving for an optimal safety action under the constraint set formed by the linear safety constraint inequalities is achieved through a quadratic programming solver of the following form: ， s.t. , ; in, It is a control variable. Let be the objective function. Represents the set of linear security constraints. and Let represent the coefficient matrix and constant vector formed by the linear security constraint inequalities, respectively; and Physical limit for the drone actuator.

7. The UAV image target navigation reinforcement learning safety control method according to claim 6, characterized in that, When the quadratic programming solver fails to solve the problem under the set of linear safety constraints, a backoff and recovery mechanism is executed, including: The optimization iteration process is interrupted and the rollback mode is switched. Execute the reverse obstacle avoidance command ,in, The preset forced reverse speed, For emergency steering angular velocity; Simultaneously assess the furthest detection distance of rays on both sides of the field of view, and guide the drone to rotate to the open side for emergency obstacle avoidance.

8. A safety control device for image target navigation reinforcement learning in unmanned aerial vehicles, characterized in that, include: The first module is used to train the navigation reinforcement learning policy network using an improved proximal policy optimization algorithm during the training phase. The improved near-end policy optimization algorithm replaces the hard truncation function with a smooth adjustment mechanism that has the continuity of the first derivative at the truncation boundary during policy update. It combines an entropy weight coefficient based on multidimensional performance feedback to construct a surrogate loss function to maintain the consistency of the feature representation of the visual encoder. The second module is used to input environmental perception information and the reference action output by the policy network of the navigation reinforcement learning into a pre-trained risk assessment network to predict the collision probability prediction value of performing the reference action. The third module is used to construct depth-driven kinematic safety constraints using the raw depth images acquired by the airborne depth sensor: a discrete observation vector is constructed by column-based minimum depth sampling. Based on the control obstacle function theory, a linear safety constraint inequality for the UAV control vector is derived; Dynamically activate corresponding constraints based on depth sample values; specifically including: The raw depth image within the field of view of the airborne depth sensor is divided into multiple sampling columns along the horizontal direction, and the minimum depth value within each sampling column is taken as the corresponding sampling ray angle. depth sample value This constitutes the discrete observation vector in polar coordinates. ; Define local barrier function ,in, This is a preset physical safety margin; Based on the planar kinematic model of the UAV, the relationship between the derivative of the local barrier function with respect to time and the linear and angular velocities of the UAV is established: ； in, , They represent respectively to , Find the first derivative. and These represent the linear velocity and angular velocity of the drone, respectively. Substituting the above relationship into the Lie derivative hard constraint of the control local barrier function, we can... To obtain information about the control variables Linear security constraint inequalities: ； in, The preset safety adjustment coefficient, Indicates the transpose operation; Set activation threshold ,in, A positive preset offset; When the sampling ray angle Corresponding depth sampling value Less than When the sampling ray is sampled, the corresponding linear safety constraint inequality is included in the linear safety constraint set of the subsequent quadratic programming solver. The fourth module is used to solve for an optimal safe action under the constraint set formed by the linear safety constraint inequalities when the collision probability prediction value exceeds the preset risk threshold, with the reference action as the optimization target, and output the optimal safe action as the actual action; otherwise, the reference action is directly output as the actual action.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the UAV image target navigation reinforcement learning safety control method as described in any one of claims 1-7.

Images